A method for controlling the dynamic range of a microphone far-field pickup amplitude

Abstract

The application relates to the technical field of audio signal processing, and discloses a microphone far-field sound pickup amplitude dynamic range control method, which comprises the following steps: processing synchronous bone conduction and air conduction signals; extracting feature vectors representing near-field interference and far-field human voice from the bone conduction signal and the air conduction signal respectively; inputting the feature vectors into an artificial intelligence decision engine for collaborative analysis to generate a gain strategy that takes into account interference suppression and human voice preservation; and adjusting the gain of the air conduction signal according to the strategy. The system comprises a signal preprocessing module, a first feature extraction module, a second feature extraction module, an artificial intelligence decision engine and a gain adjustment and reconstruction module. The application solves the gain control problem of coexistence of far-field weak human voice and near-field strong interference by using the bone conduction signal to make unambiguous observation of the near-field interference, and can effectively prevent signal clipping while ensuring sound pickup quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio signal processing, in particular to a microphone far-field pickup amplitude dynamic range control method. BACKGROUND

[0002] In the field of audio acquisition technology, especially in portable recording devices (such as AI recorders) that work in cooperation with intelligent terminal devices, it is often necessary to deal with complex pickup environments. Such devices usually use a cardioid microphone to collect distant environmental sound, such as target human voice. However, a common technical challenge is that while the cardioid microphone collects weak signals in the far field, it inevitably collects strong signals from the host device's own loudspeaker, which is extremely close.

[0003] The two sound sources differ greatly in physical location and signal amplitude, resulting in a mixed signal received by the cardioid microphone having a very wide dynamic range. This large amplitude difference poses a serious challenge to traditional gain control methods. Existing technical solutions, such as dynamic range control (DRC) or automatic gain control (AGC), usually analyze and respond to the overall amplitude of the mixed signal. These methods cannot effectively distinguish between different sound source components in the signal. When the near-field strong signal appears, the input signal may already have irreversible clipping distortion before the gain control unit responds; and if low gain is maintained at all times to avoid clipping, the target weak signal in the far field will be drowned in the system noise floor, resulting in information loss.

[0004] Therefore, the existing technology generally lacks a mechanism that can distinguish between signals from different sources in a complex acoustic environment and make forward-looking and refined gain adjustments accordingly, making it difficult to ensure that clipping distortion does not occur while simultaneously ensuring the pickup quality and clarity of the far-field target signal.

[0005] Therefore, the present application proposes a microphone far-field pickup amplitude dynamic range control method to solve the problems of the prior art. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides a microphone far-field pickup amplitude dynamic range control method, which solves the problem of improper gain control caused by the inability to effectively distinguish between the far-field target weak human voice and the near-field device strong interference sound in the mixed signal, and further causes signal clipping or low target human voice acquisition quality.

[0007] To achieve the above purpose, the present application is implemented by the following technical scheme: a microphone far-field pickup amplitude dynamic range control method, the method comprising the following steps:

[0008] S1, frame processing is performed on the bone conduction microphone signal and the air conduction microphone signal as a mixed signal which are synchronously collected, to obtain corresponding bone conduction signal frames and air conduction signal frames;

[0009] S2, based on the bone conduction signal frames, a first feature vector for characterizing near-field interference characteristics is extracted;

[0010] S3, based on the air conduction signal frames, a second feature vector for characterizing far-field voice state is extracted;

[0011] S4, the first feature vector and the second feature vector are input to a preset artificial intelligence decision engine, and the artificial intelligence decision engine is used for collaborative analysis to determine a gain control strategy package which can respond to the near-field interference characteristics and take into account the far-field voice state;

[0012] S5, according to the gain control strategy package, gain adjustment is performed on the air conduction signal frames, and an output audio signal with controlled dynamic range is reconstructed.

[0013] Preferably, in step S1, the step of frame processing the synchronously collected bone conduction microphone signal and the air conduction microphone signal as a mixed signal to obtain corresponding bone conduction signal frames and air conduction signal frames comprises:

[0014] According to a preset frame length and a preset frame shift, the bone conduction microphone signal and the air conduction microphone signal are cut to obtain bone conduction signal frames and air conduction signal frames with partial overlap in front and back;

[0015] The sample point sequence of the bone conduction signal frames and the air conduction signal frames is respectively multiplied point by point with a coefficient sequence of a smoothing window function, to suppress spectral leakage in the spectral analysis process;

[0016] The bone conduction signal frames and the air conduction signal frames after point-by-point multiplication are respectively subjected to short-time Fourier transform, to generate bone conduction spectrum data and air conduction spectrum data containing amplitude spectrum information and phase spectrum information.

[0017] Preferably, in step S2, the step of extracting a first feature vector for characterizing near-field interference characteristics based on the bone conduction signal frames comprises:

[0018] Based on the amplitude spectrum of the bone conduction spectrum data, the product of each frequency point index and the amplitude value corresponding to the frequency point index is accumulated, and the obtained sum is divided by the sum of all amplitude values of the amplitude spectrum, to determine the spectral centroid in the spectral pattern feature of the near-field interference sound type;

[0019] divide the frequency domain covered by the bone conduction frequency spectrum data into a plurality of preset subbands, and accumulate energy values of all frequency points in each of the plurality of preset subbands to obtain a sub-band energy feature representing energy frequency domain distribution of the near-field interference;

[0020] calculate a time domain envelope of the bone conduction signal frame, and extract at least one statistic based on the time domain envelope as a time domain envelope feature representing transient change characteristics of the near-field interference in the time domain;

[0021] combine the determined spectral centroid, the obtained sub-band energy feature, and the extracted time domain envelope feature into the first feature vector.

[0022] Preferably, in step S3, the step of extracting a second feature vector for representing a far-field voice state based on the air conduction signal frame comprises:

[0023] applying a voice activity detection algorithm to process the air conduction signal frame to generate a voice activity detection state representing whether there is a far-field voice component in the air conduction signal frame;

[0024] when the judgment result of the voice activity detection state is that there is a far-field voice component, applying a noise power spectrum estimation algorithm to process the air conduction frequency spectrum data to obtain an estimated noise power spectrum;

[0025] obtaining an air conduction signal total energy by accumulating energy values of all frequency points in the air conduction frequency spectrum data;

[0026] obtaining an estimated noise total energy by accumulating energy values of all frequency points in the estimated noise power spectrum;

[0027] dividing the air conduction signal total energy by the estimated noise total energy to obtain a ratio, and performing logarithmic operation on the ratio to determine an estimated signal-to-noise ratio for quantifying far-field voice intelligibility;

[0028] combining the generated voice activity detection state and the determined estimated signal-to-noise ratio into the second feature vector.

[0029] Preferably, the gain control strategy package comprises:

[0030] a multi-band target gain vector, the multi-band target gain vector containing target gain values for a plurality of preset frequency bands;

[0031] an adaptive attack time;

[0032] an adaptive release time.

[0033] Preferably, in step S4, the first feature vector and the second feature vector are input into a preset artificial intelligence decision engine, and the step of collaborative analysis by the artificial intelligence decision engine comprises:

[0034] When the voice activity detection state in the second feature vector is that there is no far-field human voice, the artificial intelligence decision engine generates a gain control strategy package with the near-field interference characteristic represented by the first feature vector as the only basis, and the gain control strategy package has a priority target of preventing signal clipping of the air conduction signal frame.

[0035] Preferably, in step S4, the first feature vector and the second feature vector are input into a preset artificial intelligence decision engine, and the step of collaborative analysis by the artificial intelligence decision engine further comprises:

[0036] When the voice activity detection state in the second feature vector is that there is far-field human voice, and the estimated signal-to-noise ratio is low, if the near-field interference intensity represented by the first feature vector is higher than a preset threshold, the artificial intelligence decision engine generates a gain control strategy package with a priority target of maintaining gain stability and suppressing output distortion, so as to avoid introducing additional noise due to drastic gain adjustment.

[0037] Preferably, in step S5, according to the gain control strategy package, the step of gain adjustment on the air conduction signal frame and reconstruction of the output audio signal with controlled dynamic range comprises:

[0038] The air conduction signal frame is decomposed into a plurality of sub-band signals by an analysis filter bank, and the sub-band signals correspond to a plurality of preset frequency bands in the gain control strategy package;

[0039] For each of the plurality of sub-band signals, a smoothed gain is applied;

[0040] All gain-applied sub-band signals are combined by a synthesis filter bank matched with the analysis filter bank to reconstruct the output audio signal.

[0041] Preferably, the step of applying a smoothed gain to each of the plurality of sub-band signals comprises:

[0042] A target gain value corresponding to the currently processed sub-band signal is obtained from the gain control strategy package;

[0043] determining a change trend of the target gain value relative to a previous time point applied smoothing gain value, and determining an adaptive time constant based on the change trend, wherein when the change trend is gain increase, the adaptive time constant takes a value of an adaptive attack time in the gain control strategy package, and when the change trend is gain decrease or keep, the adaptive time constant takes a value of an adaptive release time in the gain control strategy package;

[0044] a smoothing coefficient is calculated by the following formula

[0045]

[0046] wherein, t represents a processing time interval of a signal frame, t represents an adaptive time constant;

[0047] the smoothing gain value is updated by the following recursive formula using the smoothing coefficient

[0048]

[0049] wherein, G represents a smoothing gain value at a current time point, G represents a smoothing gain value at a previous time point, G represents a target gain value; the calculated smoothing gain value at the current time point is applied to the sub-band signal.

[0050] The application also provides a microphone far-field sound pickup amplitude dynamic range control system, the system comprising:

[0051] a signal preprocessing module, configured to perform frame processing on the synchronously collected bone conduction microphone signal and air conduction microphone signal as a mixed signal to obtain corresponding bone conduction signal frames and air conduction signal frames;

[0052] a first feature extraction module, configured to extract a first feature vector for characterizing near-field interference characteristics based on the bone conduction signal frames;

[0053] a second feature extraction module, configured to extract a second feature vector for characterizing far-field voice state based on the air conduction signal frames;

[0054] an artificial intelligence decision engine, configured to receive the first feature vector and the second feature vector for collaborative analysis, so as to determine a gain control strategy package capable of responding to the near-field interference characteristics and taking into account the far-field voice state;

[0055] an artificial intelligence decision engine, configured to receive the first feature vector and the second feature vector for collaborative analysis, so as to determine a gain control strategy package capable of responding to the near-field interference characteristics and taking into account the far-field voice state;

[0056] ​​​​​​A gain adjustment and reconstruction module is configured to perform gain adjustment on the air conduction signal frame according to the gain control strategy package, and reconstruct an output audio signal with controlled dynamic range.

[0057] The application provides a microphone far-field pickup amplitude dynamic range control method.

[0058] 1、The application introduces a bone conduction microphone signal as an independent reference for near-field interference, so that the system can accurately distinguish far-field human voice from near-field interference in the mixed signal. This mechanism based on the collaborative analysis of dual signals solves the problem of single air conduction microphone signal source ambiguity, thereby making the decision basis of dynamic range control more explicit and reliable, and reducing the gain misadjustment caused by false identification of the sound source.

[0059] 2、The application uses an artificial intelligence decision engine to perform collaborative analysis on the extracted first feature vector and second feature vector, and can dynamically generate an optimal gain control strategy package according to the real-time changing acoustic environment. This method not only considers the energy of the signal, but also comprehensively analyzes the sound type of the near-field interference, the intelligibility of the far-field human voice, and whether there is speech activity, so that the gain adjustment has scene adaptive ability and can cope with more complex actual application environment.

[0060] 3、The output gain control strategy package contains multi-dimensional control parameters, such as a multi-band target gain vector and adaptive attack and release time. This fine control strategy makes subsequent gain adjustment no longer a global rough adjustment, but can be differentiated in different frequency bands, and through smooth gain application, effectively suppresses the common pumping effect and noise breathing feeling in traditional dynamic range control, and improves the naturalness and listening quality of the final output audio.

[0061] 4、The technical solution provided by the application can generate a control strategy that prioritizes maintaining gain stability and suppressing output distortion through specific collaborative analysis logic in the face of extreme situations where strong near-field interference is superimposed on weak far-field human voice. This design avoids excessively increasing the overall gain in an attempt to amplify weak human voice, thereby preventing catastrophic amplification of noise and distortion, and ensuring the working robustness of the entire microphone far-field pickup system under poor acoustic conditions. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 The method flowchart of the application;

[0063] Figure 2 The working principle diagram of the AI recorder of the application;

[0064] Figure 3 The far-field pickup algorithm framework diagram of the application;

[0065] Figure 4 The AI recorder algorithm working principle schematic diagram of the application;

[0066] Figure 5 The system architecture diagram of the application.

[0067] Wherein, 110, signal preprocessing module; 120, first feature extraction module; 130, second feature extraction module; 140, artificial intelligence decision engine; 150, gain adjustment and reconstruction module. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the specification of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0069] Referring to Figure 1 The application provides a microphone far-field pickup amplitude dynamic range control method. In a specific embodiment, the method can include the following steps:

[0070] S1, frame processing is performed on the synchronously collected bone conduction microphone signal and air conduction microphone signal as a mixed signal to obtain corresponding bone conduction signal frames and air conduction signal frames.

[0071] S2, based on the bone conduction signal frames, a first feature vector for characterizing near-field interference characteristics is extracted.

[0072] S3, based on the air conduction signal frames, a second feature vector for characterizing far-field voice state is extracted.

[0073] S4, the first feature vector and the second feature vector are input to a preset artificial intelligence decision engine, and the artificial intelligence decision engine is used for collaborative analysis to determine a gain control strategy package that can respond to near-field interference characteristics and take into account far-field voice state.

[0074] S5, gain adjustment is performed on the air conduction signal frames according to the gain control strategy package, and a dynamic range controlled output audio signal is reconstructed.

[0075] Referring to Figure 5 To implement the above method, the application further provides a microphone far-field pickup amplitude dynamic range control system. The system includes:

[0076] The signal pre-processing module 110, the first feature extraction module 120, the second feature extraction module 130, the artificial intelligence decision engine 140 and the gain adjustment and reconstruction module 150.

[0077] The signal pre-processing module 110 is responsible for receiving and processing the synchronized bone conduction and air conduction microphone signals.

[0078] The first feature extraction module 120 and the second feature extraction module 130 extract feature vectors from the processed bone conduction and air conduction signals respectively.

[0079] The artificial intelligence decision engine 140 receives the two kinds of feature vectors, performs collaborative analysis and outputs a gain control strategy.

[0080] The gain adjustment and reconstruction module 150 adjusts the air conduction signal according to the strategy and outputs the result.

[0081] Reference Figure 1 The method flow shown in and Figure 5 The system structure shown in, the specific implementation steps of the method of the present application are described.

[0082] First, in step S1, the pre-processing operation of the signal is performed. The operation is completed by the Figure 5 The input of the signal pre-processing module 110 is two parallel, synchronized, time-domain digital signals, i.e. bone conduction microphone signals and air conduction microphone signals.

[0083] In order to facilitate time-frequency analysis, the signal pre-processing module 110 first performs frame processing on the two continuous signal streams. According to a pre-set frame length and a pre-set frame shift, the continuous signal is cut into a series of data frames with partial overlap between adjacent frames. Thus, a discrete bone conduction signal frame sequence and an air conduction signal frame sequence are obtained.

[0084] Before performing spectral transformation on the signal frames, in order to suppress the spectral leakage inevitably introduced by signal frame truncation, a smoothing window function needs to be applied to each bone conduction signal frame and air conduction signal frame. In a specific embodiment, a Hanning window function can be used. The application process is to multiply the coefficient sequence of the window function with the sample point sequence of the signal frame point by point.

[0085] Subsequently, the signal pre-processing module 110 performs short-time Fourier transform (STFT) on each bone conduction signal frame and air conduction signal frame after applying the smoothing window function. The short-time Fourier transform maps each frame of time-domain signal to a complex frequency domain, and the result contains not only the amplitude spectrum information of the signal in that time period, but also the phase spectrum information. The transformation is usually realized by fast Fourier transform (FFT) algorithm to improve the operation efficiency.

[0086] After the short-time Fourier transform is completed, the output obtained is the bone conduction spectrum data and the air conduction spectrum data. These two spectrum data provide the necessary frequency domain information basis for the feature extraction in the subsequent steps, and are transmitted to the first feature extraction module 120 and the second feature extraction module 130, respectively.

[0087] In step S2, the extraction of the first feature vector is performed. This step is performed by the first feature extraction module 120 shown in the figure, and the input is the bone conduction signal frame and the bone conduction spectrum data from the signal preprocessing module 110. The purpose of this step is to calculate and combine a first feature vector for comprehensively characterizing the near-field interference characteristics from these data. In a specific embodiment, the first feature vector is composed of several different dimension features. Figure 5 The first type of feature is the spectral morphology feature, which is used to characterize the sound type attribute of the near-field interference. This type of feature is calculated based on the bone conduction spectrum data, and in an embodiment, can include the spectral centroid and the spectral flatness. Among them, the spectral centroid is used to quantify the center of gravity frequency position of the spectral energy. Specifically, the first feature extraction module 120 calculates the spectral centroid according to the amplitude spectrum of the input bone conduction spectrum data through the following formula:

[0088]

[0089] ;

[0090] In the formula: represents the spectral centroid value of the current bone conduction signal frame; represents the amplitude value of the bone conduction spectrum data at the th frequency point; is the frequency point index, whose value ranges from 1 to ; is the total number of frequency points, which is determined by the number of points in the short-time Fourier transform in step S1.

[0091] In addition, the spectral flatness can also be calculated to measure the degree to which the signal spectrum approaches a tone or noise.

[0092] The second type of feature is the sub-band energy feature, which is used to characterize the energy distribution of the near-field interference in the frequency domain. To calculate this feature, the first feature extraction module 120 first divides the entire frequency domain into multiple sub-bands according to the preset frequency point boundary. Then, for each sub-band, the total energy of the sub-band is calculated by accumulating the energy values (i.e. the square of the amplitude values) of the bone conduction spectrum data at all frequency points in the sub-band. The energy values of all sub-bands together constitute the sub-band energy feature.

[0093] ​The third type of feature is the time-domain envelope feature, which is used to characterize the transient variation of the near-field interference in time domain. The calculation of this feature is based on the bone-conduction signal frame in time domain. The first feature extraction module 120 first calculates the time-domain envelope of the bone-conduction signal frame, and then extracts one or more statistics from the envelope that can reflect its transient variation, such as the peak value, mean value or zero-crossing rate of the envelope.

[0094] The above-mentioned extracted spectral morphology feature, frequency-band energy feature and time-domain envelope feature are combined together to form a first feature vector. This vector is then transmitted to the artificial intelligence decision engine 140 as an important input for its collaborative analysis.

[0095] In step S3, the extraction of the second feature vector is performed. This step is performed by the second feature extraction module 130 shown in Figure 5 , which takes as input the air-conduction signal frame and the air-conduction spectral data from the signal preprocessing module 110. The purpose of this step is to calculate and combine a second feature vector from these mixed signal data to characterize the state of the far-field human voice.

[0096] In a specific embodiment, the extraction of the second feature vector includes the following processing procedures. First, the second feature extraction module 130 applies a voice activity detection (VAD) algorithm to the input air-conduction signal frame. This algorithm analyzes multiple acoustic characteristics of the signal to generate a binary or multi-valued voice activity detection state. This state is used to determine whether there is a far-field human voice component in the current air-conduction signal frame.

[0097] Second, the second feature extraction module 130 will start the subsequent calculation only when the voice activity detection state determines that there is a far-field human voice. Based on the air-conduction spectral data, the module applies a noise power spectrum estimation algorithm to estimate the power spectrum of the noise component in the current signal frame. This estimation process can be performed on multiple consecutive frames to find the minimum value or slowly changing part of the signal energy through statistical analysis, in order to obtain a continuous tracking of the background noise level.

[0098] After obtaining the estimated noise power spectrum, the second feature extraction module 130 calculates an estimated signal-to-noise ratio for quantifying the intelligibility of the far-field human voice. The estimated signal-to-noise ratio is calculated by the following formula:

[0099] ;

[0100] In the formula: represents the estimated signal-to-noise ratio of the current air-conduction signal frame, with the unit of decibel (dB); represents the complex value of the air-conduction spectral data at the th frequency point; represents the complex value of the air-conduction signal at the The energy at each frequency point; Represents the noise power spectrum obtained by the noise power spectrum estimation algorithm. The complex value of the noise spectrum estimated at the frequency point; Represents the estimated noise in The energy at each frequency point; is the frequency index, and its value range is from 1 to ; is the total number of frequency points, which is determined by the number of points of the short-time Fourier transform in step S1.

[0101] Finally, the second feature extraction module 130 combines the generated voice activity detection status and the estimated signal-to-noise ratio calculated under specific conditions into a second feature vector. This vector is then transmitted to the artificial intelligence decision engine 140 as another input for collaborative analysis. If the voice activity detection status of the current frame indicates that far-field human voice is absent, the estimated signal-to-noise ratio may be set to a preset invalid value or not calculated or transmitted.

[0102] In step S4, artificial intelligence decision making and gain strategy generation are performed. Figure 5 The artificial intelligence decision engine 140 is shown as complete. The input of the artificial intelligence decision engine 140 is connected to the output of the first feature extraction module 120 and the second feature extraction module 130, respectively, to receive the first feature vector and the second feature vector. Its function is to perform a collaborative analysis based on these two input feature vectors and output a gain control strategy package.

[0103] In one specific embodiment, the gain control strategy package is a data structure consisting of three core components: a multi-band target gain vector, an adaptive attack time, and an adaptive release time. The multi-band target gain vector contains target gain values ​​for multiple preset frequency bands, enabling frequency-selective gain adjustment of the signal. The adaptive attack time and adaptive release time are used to control the speed of subsequent gain smoothing.

[0104] The AI ​​decision engine 140 executes a rule-based or pre-trained model-based decision process to generate the most appropriate gain control strategy package based on the real-time acoustic scenario reflected by the input feature vector. This collaborative analysis process includes processing the following typical logical scenarios:

[0105] One logical scenario is when the voice activity detection status in the second feature vector is no far-field voice present, indicating that the dominant component in the current air-conducted signal is near-field interference or environmental noise. In this case, the decision process of the artificial intelligence decision engine 140 will rely solely on the near-field interference characteristics represented by the first feature vector. The gain control policy package it generates will prioritize preventing signal clipping of the air-conducted signal frame, for example, by setting a lower or suppressive target gain value according to the energy feature in the first feature vector.

[0106] Another logical scenario is when the voice activity detection status in the second feature vector is far-field voice present, and its included estimated signal-to-noise ratio is low, if the near-field interference strength represented by the first feature vector is higher than a pre-set threshold at this time, it indicates that the current scenario is strong near-field interference superimposed with weak far-field voice. In this case, if the gain is greatly increased to try to enhance the far-field voice, it will disproportionately amplify the stronger near-field interference, resulting in severe distortion of the output signal. Therefore, the artificial intelligence decision engine 140 will generate a gain control policy package that prioritizes maintaining stable gain and suppressing output distortion. The target gain value in the strategy package will tend to be maintained at the level of the last frame or slightly attenuated, while setting a longer adaptive attack and release time to avoid introducing additional noise or pumping effect in the hearing sensation due to dramatic gain adjustment.

[0107] For other scenario combinations, such as no near-field interference but clear far-field voice present, the artificial intelligence decision engine 140 will generate a gain control policy package that targets to boost the amplitude of the far-field voice signal.

[0108] Finally, the artificial intelligence decision engine 140 will transmit the gain control policy package generated according to the result of each collaborative analysis to the gain adjustment and reconstruction module 150 to guide subsequent gain adjustment operations.

[0109] In step S5, gain adjustment and signal reconstruction are performed. This step is completed by the gain adjustment and reconstruction module 150 shown in Figure 5 The input end of this module is connected with the artificial intelligence decision engine 140 and the signal preprocessing module 110 respectively, to receive the gain control policy package and the original air-conducted signal frame.

[0110] In a specific embodiment, the gain adjustment and reconstruction module 150 first decomposes the air-conducted signal frame into multiple sub-band signals through an analysis filter bank. The frequency band division manner of the analysis filter bank corresponds to the frequency band division of the multi-frequency segment target gain vector in the gain control policy package, to ensure that each sub-band signal can find a corresponding target gain value.

[0111] For each subband signal, the gain adjustment and reconstruction module 150 obtains the target gain value corresponding to the subband from the gain control strategy package. To prevent the gain mutation and unnatural hearing that can be introduced by directly applying the target gain value to the subband signal, the target gain value needs to be smoothed.

[0112] The smoothing process first needs to determine the change trend of the target gain value relative to the smoothed gain value already applied at the previous time. Based on the change trend, a smoothing coefficient is calculated.

[0113] If the current target gain value is greater than the smoothed gain value at the previous time, it is determined that the gain is increasing, and the smoothing coefficient is calculated based on the adaptive attack time in the gain control strategy package.

[0114] If the current target gain value is less than or equal to the smoothed gain value at the previous time, it is determined that the gain is decreasing or maintaining, and the smoothing coefficient is calculated based on the adaptive release time in the gain control strategy package.

[0115] The specific calculation method of the smoothing coefficient is as follows:

[0116] ;

[0117] In the formula: represents the calculated smoothing coefficient; represents the processing time interval of the signal frame, i.e., the number of seconds corresponding to the frame shift; represents the adaptive time constant. When in the gain increasing trend, the adaptive attack time in the gain control strategy package is taken as the value; when in the gain decreasing or maintaining trend, the adaptive release time is taken as the value.

[0118] After the smoothing coefficient is calculated, the gain adjustment and reconstruction module 150 uses the coefficient to smooth the target gain value by the following recursive formula to generate a smoothly changing gain:

[0119] ;

[0120] In the formula: represents the smoothed gain value calculated at the current time and to be applied to the subband signal; represents the smoothed gain value already calculated and applied at the previous time; represents the target gain value at the current time obtained from the gain control strategy package; is the smoothing coefficient calculated according to the above method.

[0121] The gain adjustment and reconstruction module 150 applies the calculated smoothing gain of each subband to the corresponding subband signal respectively. After the gain application of all subbands is completed, all gain-adjusted subband signals are merged by a synthesis filter bank matched with the analysis filter bank to reconstruct a frame of output audio with controlled dynamic range.

[0122] Finally, the final continuous output audio signal with controlled dynamic range can be obtained by splicing the continuous output audio frames (for example, using the overlap-add method).

[0123] To further illustrate the implementation process and technical effects of the technical solutions of the present application, a specific application scenario is provided below as an embodiment.

[0124] Referring to Figures 2-5 , an application scenario is assumed: a user uses an AI recorder integrated with the microphone far-field sound pickup amplitude dynamic range control system of the present application in a conference room. The recorder is placed on the conference table or in contact with the back plate of the mobile phone, and the target is to record the speech of a speaker 3 meters away (i.e. far-field voice). During the recording process, the speaker of the host device (smartphone) suddenly plays a short but loud prompt tone (i.e. near-field interference sound).

[0125] In this scenario, the complete workflow of the system of the present application is as follows:

[0126] Before the prompt tone is played, the system only collects the far-field voice. The signal preprocessing module 110 frames and performs short-time Fourier transform on the synchronous air conduction and bone conduction microphone signals. Since there is no speaker vibration at this time, the first feature vector extracted by the first feature extraction module 120 from the bone conduction signal reflects very low near-field interference energy. At the same time, the second feature extraction module 130 detects the presence of far-field voice (the speech activity detection state is true) from the air conduction signal and calculates an estimated signal-to-noise ratio at a medium level. The artificial intelligence decision engine 140 receives the two sets of feature vectors, analyzes them cooperatively, and determines that it is a far-field voice scene without interference, and then generates a gain control strategy package targeting at improving the intelligibility of the voice, which contains a higher multi-band target gain value. The gain adjustment and reconstruction module 150 applies a higher gain to the air conduction signal according to the strategy package, so that the output speaker's voice is clear and audible.

[0127] ​When the prompt tone suddenly comes, the bone conduction microphone picks up strong housing vibration signal, and the air conduction microphone picks up the mixed signal of strong prompt tone and weak human voice. At this time, the first feature extraction module 120 extracts the first feature vector which immediately reflects the high-intensity near-field interference characteristics. The second feature extraction module 130 can still detect human voice activity, but the estimated signal-to-noise ratio calculated will drop sharply. The artificial intelligence decision engine 140 receives these two sets of significantly changed feature vectors, and after collaborative analysis, immediately determines the scene as weak far-field human voice under strong near-field interference. To prevent signal clipping, the artificial intelligence decision engine 140 will immediately generate a new gain control strategy package, and the multi-band target gain value in the strategy package is set to a low value with a suppression effect, and may be accompanied by a short adaptive attack time to achieve fast response. The gain adjustment and reconstruction module 150 receives this new strategy, and its internal gain smoothing processing mechanism quickly reduces the applied smoothing gain, thereby effectively suppressing the amplitude of the prompt tone and avoiding clipping distortion of the output signal.

[0128] After the prompt tone ends, the system state returns. The first feature extraction module 120 again outputs a feature vector with low interference energy, and the artificial intelligence decision engine 140 re-generates a target gain strategy that enhances human voice. The gain adjustment and reconstruction module 150 again smoothly restores the gain to a higher level, and continues to clearly pick up far-field human voice.

[0129] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for controlling the dynamic range of far-field sound pickup amplitude of a microphone, characterized in that: The method comprises the following steps: S1. performing frame processing on the synchronously collected bone conduction microphone signal and the air conduction microphone signal as a mixed signal to obtain corresponding bone conduction signal frames and air conduction signal frames; S2. Extracting a first eigenvector for characterizing near-field interference characteristics based on the bone conduction signal frame; S3. Extracting a second eigenvector for characterizing a far-field human voice state based on the air conduction signal frame; S4. Inputting the first feature vector and the second feature vector into a preset artificial intelligence decision engine, and having the artificial intelligence decision engine perform collaborative analysis to determine a gain control strategy package that can respond to the near-field interference characteristics and take into account the far-field human voice state; S5. Perform gain adjustment on the air conduction signal frame according to the gain control strategy package, and reconstruct an output audio signal with controlled dynamic range.

2. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 1, wherein: In step S1, the steps of performing frame processing on the synchronously collected bone conduction microphone signal and the air conduction microphone signal as a mixed signal to obtain corresponding bone conduction signal frames and air conduction signal frames include: The bone conduction microphone signal and the air conduction microphone signal are segmented according to a preset frame length and a preset frame shift to obtain bone conduction signal frames and air conduction signal frames that partially overlap. multiplying the sampling point sequences of the bone conduction signal frame and the air conduction signal frame by a coefficient sequence of a smoothing window function point by point, respectively, to suppress spectrum leakage during spectrum analysis; The bone conduction signal frame and the air conduction signal frame after point-by-point multiplication are respectively subjected to short-time Fourier transform to generate bone conduction spectrum data and air conduction spectrum data containing amplitude spectrum information and phase spectrum information.

3. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 2, wherein: In step S2, the step of extracting a first eigenvector for characterizing near-field interference characteristics based on the bone conduction signal frame includes: Determine, based on the amplitude spectrum of the bone conduction spectrum data, a spectrum centroid in the spectrum morphological feature of the near-field interference sound type by accumulating the product of each frequency index and the amplitude value corresponding to the frequency index, and dividing the accumulated sum by the sum of all amplitude values ​​of the amplitude spectrum; Dividing the frequency domain covered by the bone conduction spectrum data into a plurality of preset sub-bands, and accumulating the energy values ​​of all frequency points in each of the plurality of preset sub-bands to obtain a frequency-band energy feature representing the energy frequency domain distribution of the near-field interference; Calculating a time domain envelope of the bone conduction signal frame, and extracting at least one statistic based on the time domain envelope as a time domain envelope feature characterizing a transient change characteristic of the near-field interference in the time domain; The determined spectrum centroid, the obtained frequency segment energy features, and the extracted time domain envelope features are combined into the first feature vector.

4. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 2, wherein: In step S3, the step of extracting a second eigenvector for characterizing the far-field human voice state based on the air conduction signal frame includes: Applying a voice activity detection algorithm to process the air conduction signal frame to generate a voice activity detection state indicating whether a far-field human voice component is present in the air conduction signal frame; When the voice activity detection state determines that a far-field human voice component exists, applying a noise power spectrum estimation algorithm to process the air conduction spectrum data to obtain an estimated noise power spectrum; The total energy of the air conduction signal is obtained by accumulating the energy values ​​of all frequency points in the air conduction spectrum data; The estimated total noise energy is obtained by accumulating the energy values ​​of all frequency points in the estimated noise power spectrum; Dividing the total energy of the air conduction signal by the estimated total energy of the noise to obtain a ratio, and performing a logarithmic operation on the ratio to determine an estimated signal-to-noise ratio for quantifying far-field vocal clarity; The generated voice activity detection state and the determined estimated signal-to-noise ratio are combined into the second feature vector.

5. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 1, wherein: The gain control strategy package includes: a multi-band target gain vector, wherein the multi-band target gain vector includes target gain values ​​for a plurality of preset frequency bands; Adaptive attack time; Adaptive release time.

6. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 5, characterized in that: In step S4, the first feature vector and the second feature vector are input into a preset artificial intelligence decision engine, and the artificial intelligence decision engine performs collaborative analysis, including: When the voice activity detection status in the second eigenvector is that there is no far-field human voice, the artificial intelligence decision engine uses the near-field interference characteristics represented by the first eigenvector as the sole basis to generate a gain control strategy package with the priority goal of preventing signal clipping from occurring in the air conduction signal frame.

7. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 5, characterized in that: In step S4, the first feature vector and the second feature vector are input into a preset artificial intelligence decision engine, and the step of collaborative analysis by the artificial intelligence decision engine further includes: When the voice activity detection status in the second feature vector indicates the presence of far-field human voice and the estimated signal-to-noise ratio is lower than a preset signal-to-noise ratio threshold, if the near-field interference intensity represented by the first feature vector is higher than a preset threshold, the artificial intelligence decision engine generates a gain control strategy package with the priority goals of maintaining gain stability and suppressing output distortion, so as to avoid the introduction of additional noise due to drastic gain adjustment.

8. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 1, wherein: In step S5, the steps of performing gain adjustment on the air conduction signal frame according to the gain control strategy package and reconstructing an output audio signal with controlled dynamic range include: Decomposing the air conduction signal frame into a plurality of sub-band signals by an analysis filter bank, wherein the sub-band signals correspond to a plurality of preset frequency bands in the gain control strategy package; Applying a smoothed gain to each of the plurality of sub-band signals; All sub-band signals to which gains have been applied are combined by a synthesis filter bank matched with the analysis filter bank to reconstruct the output audio signal.

9. The method for controlling the dynamic range of far-field sound pickup amplitude of a microphone according to claim 8, characterized in that: The step of applying a smoothed gain to each of the plurality of sub-band signals comprises: Obtaining a target gain value corresponding to a currently processed subband signal from the gain control strategy package; Determining a change trend of the target gain value relative to a smoothed gain value applied at a previous moment, and determining an adaptive time constant based on the change trend, wherein when the change trend is a gain increase, the adaptive time constant is set to an adaptive attack time in the gain control strategy package; and when the change trend is a gain decrease or gain retention, the adaptive time constant is set to an adaptive release time in the gain control strategy package; A smoothing coefficient is calculated by the following formula : ； Where, Represents the processing time interval of the signal frame, represents the adaptive time constant; By the following recursive formula, using the smoothing coefficient Update the smoothing gain value: ； Where, Represents the smoothing gain value at the current moment, Represents the smoothing gain value of the previous moment, represents the target gain value; The calculated smoothing gain value at the current moment Applied to the sub-band signal.

10. A far-field sound pickup amplitude dynamic range control system for a microphone, applied to the method according to any one of claims 1 to 9, characterized in that: The system comprises: A signal preprocessing module is used to perform frame processing on the synchronously collected bone conduction microphone signal and the air conduction microphone signal as a mixed signal to obtain corresponding bone conduction signal frames and air conduction signal frames; A first feature extraction module is configured to extract a first feature vector for characterizing near-field interference characteristics based on the bone conduction signal frame; A second feature extraction module is used to extract a second feature vector for characterizing a far-field human voice state based on the air conduction signal frame; an artificial intelligence decision engine, configured to receive the first feature vector and the second feature vector for collaborative analysis to determine a gain control strategy package that can respond to the near-field interference characteristics and take into account the far-field human voice state; The gain adjustment and reconstruction module is used to perform gain adjustment on the air conduction signal frame according to the gain control strategy package, and reconstruct an output audio signal with controlled dynamic range.

Images