Directional microphone array voice distortion detection, repair method, device and system

Abstract

The application discloses a directional microphone array voice distortion detection and repair method, device and system. The application calculates signal energy before post-filter processing based on a current frame frequency domain signal input to a post-filter of a directional microphone array. The current frame frequency domain signal can be a frequency domain signal of a target direction after beam pointing is strengthened by an adaptive beam forming algorithm. Further, signal energy after post-filter processing can be calculated based on a post-filter output post-filter signal. The voice quality ratio of the current frame is calculated based on the signal energy after post-filter processing and the signal energy before post-filter processing. The distortion degree of the current frame can be obtained by compression and expansion of the voice quality ratio in a set manner, which can be used to measure the distortion size of the post-filter signal of the current frame, that is, to measure the distortion condition of the post-filter signal obtained by processing the current frame frequency domain signal by the post-filter.

Description

Technical Field

[0001] This application relates to the field of microphone array pickup technology, and more specifically, to a method, apparatus and system for detecting and repairing speech distortion in a directional microphone array. Background Technology

[0002] With the continuous development of social technology, various devices and services based on voice interaction are rapidly becoming widespread. Taking directional microphone arrays as an example, existing directional microphone array pickup technology generally first uses a fixed beamforming algorithm to extract speech from several directions, then uses an adaptive beamforming algorithm to enhance beam directionality, and finally uses a post-filter to further suppress noise in the speech from the target direction and interference from speech from other directions, thus significantly improving the speech signal-to-noise ratio.

[0003] Existing solutions perform well in relatively good acoustic environments with fixed speaker positions and low reverberation. However, in real-world scenarios, the speaker's position is not always at the theoretical target location, and the environment is highly uncertain. This can lead to leakage of the target direction speech signal in other beam directions, causing significant distortion in the target speech after post-filtering. Distorted speech signals significantly reduce the accuracy of backend speech recognition.

[0004] Therefore, it is necessary to provide a distortion detection scheme for the speech signal of a directional microphone array to evaluate the distortion of the post-filtered signal output by the post-filter in the microphone array. Summary of the Invention

[0005] In view of the above problems, this application is proposed to provide a method, apparatus, and system for detecting and repairing speech distortion in a directional microphone array, so as to realize distortion detection of the post-filtered signal output by the post-filter in the directional microphone array. The specific solution is as follows:

[0006] Firstly, a method for detecting speech distortion in a directional microphone array is provided, used to detect distortion in the post-filtered signal output by a post-filter in a directional microphone array. The method includes:

[0007] Calculate the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter;

[0008] Based on the post-filtered signal output by the post-filter, calculate the signal energy after post-filtering.

[0009] The speech quality ratio of the current frame is calculated based on the signal energy after post-filtering and the signal energy before post-filtering.

[0010] The speech quality ratio of the current frame is companded according to a set method to obtain the distortion of the current frame. The distortion is used to measure the amount of distortion of the post-filtered signal of the current frame.

[0011] Secondly, a method for restoring speech distortion in a directional microphone array is provided, which is used to restore the distortion of the post-filtered signal output by the post-filter in the directional microphone array. The method includes:

[0012] Obtain the distortion of the post-filter signal of the current frame output by the post-filter obtained by the above distortion detection method;

[0013] Based on the distortion, the current frame frequency domain signal input to the post-filter is mixed with the post-filter signal to obtain a mixed frequency domain signal;

[0014] The mixed frequency domain signal is subjected to inverse Fourier transform to obtain the time domain speech signal output.

[0015] Thirdly, a directional microphone array speech distortion detection device is provided, used to detect distortion in the post-filtered signal output by the post-filter in the directional microphone array. The device includes:

[0016] The first energy calculation unit is used to calculate the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter.

[0017] The second energy calculation unit is used to calculate the energy of the signal after post-filtering based on the post-filtered signal output by the post-filter.

[0018] The speech quality ratio calculation unit is used to calculate the speech quality ratio of the current frame based on the signal energy after the post-filtering process and the signal energy before the post-filtering process.

[0019] The distortion calculation unit is used to compress and expand the speech quality ratio of the current frame according to a set method to obtain the distortion of the current frame. The distortion is used to measure the distortion of the post-filtered signal of the current frame.

[0020] Fourthly, a directional microphone array speech distortion repair device is provided for repairing distortion in the post-filtered signal output by the post-filter in a directional microphone array. The device includes:

[0021] The distortion acquisition unit is used to acquire the distortion of the post-filter signal of the current frame output by the post-filter obtained by the aforementioned distortion detection method.

[0022] A signal mixing unit is used to mix the current frame frequency domain signal input to the post-filter with the post-filter signal based on the distortion level to obtain a mixed frequency domain signal;

[0023] The signal transformation unit is used to perform inverse Fourier transform on the mixed frequency domain signal to obtain a time-domain speech signal output.

[0024] Fifthly, a directional microphone array speech distortion repair system is provided, comprising:

[0025] A fixed beamforming unit is used to extract speech from several directions using a fixed beamforming algorithm;

[0026] An adaptive beamforming unit is used to enhance the beam pointing of speech extracted from several directions by a fixed beamforming unit using an adaptive beamforming algorithm, thereby obtaining the frequency domain signal of the target direction and the frequency domain signal of other directions.

[0027] A post-filter is used to perform post-filtering processing on the signal output by the adaptive beamforming unit to obtain a post-filtered signal.

[0028] The distortion detection unit is used to calculate the distortion degree of the post-filtered signal of the current frame output by the post-filter using the above-described distortion detection method.

[0029] The distortion repair unit is used to mix the frequency domain signal in the target direction with the post-filtered signal based on the distortion degree to obtain a mixed frequency domain signal; and to perform an inverse Fourier transform on the mixed frequency domain signal to obtain a time-domain speech signal output.

[0030] Using the above technical solution, this application calculates the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter. The current frame frequency domain signal can be the frequency domain signal of the target direction after the beam pointing is enhanced by an adaptive beamforming algorithm. Furthermore, based on the post-filtered signal output by the post-filter, the signal energy after post-filtering can be calculated. Based on the signal energy after post-filtering and the signal energy before post-filtering, the speech quality ratio of the current frame is calculated. The speech quality ratio can be compressed according to a set method to obtain the distortion of the current frame, which can be used to measure the distortion of the post-filtered signal of the current frame, that is, to measure the distortion of the post-filtered signal obtained after the current frame frequency domain signal is processed by the post-filter. Attached Figure Description

[0031] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0032] Figure 1This is a schematic flowchart of a directional microphone array speech distortion detection method provided in an embodiment of this application;

[0033] Figure 2 An example is a schematic diagram of an existing directional microphone array pickup system.

[0034] Figure 3 An example is provided, illustrating the relationship between speech quality ratio and distortion variation curves.

[0035] Figure 4 This is a flowchart illustrating a method for restoring speech distortion using a directional microphone array, as provided in an embodiment of this application.

[0036] Figure 5 A schematic diagram of a directional microphone array speech distortion repair system is provided.

[0037] Figure 6 A schematic diagram of a directional microphone array speech distortion detection device provided in this application embodiment;

[0038] Figure 7 This is a schematic diagram of a directional microphone array voice distortion repair device provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] This application provides a speech distortion detection scheme for directional microphone arrays, applicable to distortion detection of post-filtered signals output from post-filters in various types of directional microphone arrays. Based on this, subsequent tasks, such as distortion repair, can be performed on the obtained post-filtered signal according to its distortion level.

[0041] The solution proposed in this application can be implemented based on a terminal with data processing capabilities. This terminal can be a directional microphone array device, a back-end processing device connected to the directional microphone array, etc.

[0042] Next, combined Figure 1 The directional microphone array speech distortion detection method of this application may include the following steps:

[0043] Step S100: Calculate the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter.

[0044] Reference Figure 2 , Figure 2 A schematic diagram of an existing directional microphone array pickup system is provided.

[0045] First, using a fixed beamforming unit and a fixed beamforming algorithm, the speech signals collected by each directional microphone in the directional microphone array are used. Figure 2 The example illustrates a scenario with four microphones (Mic1-Mic4) to extract speech from several directions. Figure 2 The example b0 to bn are sent to the GSC unit. The GSC unit uses the generalized sidelobe cancellation GSC algorithm to enhance beam pointing, and obtains the target direction speech signal Target_b0 and other direction speech signals Ref_b1 to bn. These are then sent to the post-filter for post-filtering processing, and the post-filtered signal pf_output is output.

[0046] The GSC unit can be replaced with other adaptive beamforming algorithm units to enhance beam pointing using other adaptive beamforming algorithms.

[0047] by Figure 2 Taking the example of the sound pickup system, in this step, the target direction speech signal Target_b0 obtained by the GSC unit can be used as the current frame frequency domain signal to calculate its signal energy before post-filtering.

[0048] Step S110: Calculate the signal energy after post-filtering based on the post-filter output signal.

[0049] Step S120: Calculate the speech quality ratio of the current frame based on the signal energy after post-filtering and the signal energy before post-filtering.

[0050] Specifically, the ratio of the signal energy after post-filtering to the signal energy before post-filtering can be calculated, and the result is used as the speech quality ratio of the current frame. In this step, the speech signal energy is calculated as the speech signal quality, thereby obtaining the quality ratio of the speech signal after post-filtering to that before processing.

[0051] Step S130: Compare the speech quality ratio of the current frame according to a set method to obtain the distortion of the current frame. The distortion is used to measure the distortion of the post-filtered signal of the current frame.

[0052] Specifically, this application can compress and expand the speech quality ratio of the current frame according to a set distortion range, so as to compress and expand it into the distortion range to obtain the distortion of the current frame. Overall, the speech quality ratio and the distortion are positively correlated.

[0053] The directional microphone array speech distortion detection method provided in this application calculates the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter. The current frame frequency domain signal can be the frequency domain signal of the target direction after the beam pointing is enhanced by an adaptive beamforming algorithm. Furthermore, based on the post-filtered signal output by the post-filter, the signal energy after post-filtering can be calculated. Based on the signal energy after post-filtering and the signal energy before post-filtering, the speech quality ratio of the current frame is calculated. The speech quality ratio is then compressed and expanded according to a set method to obtain the distortion of the current frame, which can be used to measure the distortion of the post-filtered signal of the current frame, that is, to measure the distortion of the post-filtered signal obtained after the current frame frequency domain signal is processed by the post-filter.

[0054] In some embodiments of this application, the process of calculating the signal energy before post-filtering in step S100 and the process of calculating the signal energy after post-filtering in step S110 are described.

[0055] Specifically, according to the beamforming principle, the directivity of low frequencies is relatively poor, while the mid-frequency and some high-frequency frequencies contain richer information. Therefore, the above step S100 may specifically include: calculating the sum of the energy of each frequency point in the set mid-frequency band in the current frame frequency domain signal, as the signal energy before post-filtering processing.

[0056] Specifically, step S110 may include: calculating the sum of the energy of each frequency point in the set mid-frequency band in the post-filtered signal output by the post-filter, as the signal energy after post-filtering.

[0057] The intermediate frequency band can be either the intermediate frequency band or the intermediate frequency band plus a portion of the high frequency band. For example, the intermediate frequency band can be set to 1500Hz to 5000Hz.

[0058] Based on the above Figure 2 Take the example of a sound pickup system:

[0059] The current frame frequency domain signal can be represented as Gsc_b0, and the post-filter signal can be represented as pf_output. Therefore:

[0060] The signal energy GscP before post-filtering and the signal energy PfP after post-filtering are expressed as follows:

[0061]

[0062]

[0063] Here, abs() represents calculating the signal energy, and k represents the frequency index.

[0064] The frequency domain signal of the current frame input to the post-filter and the post-filter signal can be signals obtained at a sampling rate of 16000 Hz and a 512-point FFT. Considering the short-term stationary nature of speech, at a sampling rate of 16000 Hz and a 512-point FFT, the sampled data is (512 / 16000 = 32 milliseconds), which conforms to the short-term stationary nature of speech. Of course, the sampling rate can also be selected as 8000 Hz or other values, and the FFT frequency point can also be selected as 128 or 256, etc.

[0065] With a sampling rate of 16000Hz and a 512-point FFT, when the intermediate frequency band is set to 1500Hz to 5000Hz, the frequency point corresponding to 1500Hz is: 1500 / (16000 / 512) = 48, and the frequency point corresponding to 5000Hz is: 5000 / (16000 / 512) = 160.

[0066] In some embodiments of this application, considering that although speech signals are short-term stationary but long-term non-stationary, and the duration of each frame processed in this application is relatively short, many frames in a continuous sentence may have low energy, which may be mistaken for non-speech signals. Therefore, in this embodiment, before calculating the speech quality ratio of the current frame based on the signal energy before and after the post-filtering process in step S120, a regression smoothing process can be added to the calculated signal energy before and after the post-filtering process. Specifically:

[0067] The signal energy GscP before post-filtering of the current frame can be subjected to regression smoothing with historical values ​​to obtain the smoothed signal energy before post-filtering.

[0068] The formula is expressed as follows:

[0069] GscSmP i =GscSmP i-1 *0.9+GscP*0.1

[0070] Among them, GscSmP i GscSmP represents the signal energy before post-filtering processing after smoothing of the current frame. i-1 This represents the signal energy before post-filtering after smoothing the previous frame. The smoothing factor can be 0.1 or other values.

[0071] Similarly, the post-filtered signal energy PfP of the current frame can be subjected to regression smoothing with historical values. The smoothing factor can be 0.1 or other values ​​to obtain the smoothed post-filtered signal energy.

[0072] The formula is expressed as follows:

[0073] PfSmP i =PfSmP i-1 *0.9+PfP*0.1

[0074] Among them, PfSmP i PfSmP represents the signal energy after post-filtering and smoothing of the current frame. i-1 This represents the signal energy after post-filtering processing following the smoothing of the previous frame. The smoothing factor can take values ​​of 0.1 or other values.

[0075] Based on the regression smoothing process performed on the signal energy before and after post-filtering, the process of calculating the speech quality ratio of the current frame in step S120 can specifically include:

[0076] Based on the signal energy after smoothing and post-filtering, and the signal energy before smoothing and post-filtering, the speech quality ratio of the current frame is calculated using the following formula:

[0077] RatioP i =PfSmP i / GscSmP i

[0078] Among them, RatioP i This indicates the audio quality ratio of the current frame.

[0079] The solution provided in this embodiment adds a process of regression smoothing for the calculated signal energy before and after post-filtering. Through regression smoothing, it can be ensured that signal frames with smaller energy in the middle of a continuous sentence will not be mistaken for non-speech signals, that is, the phenomenon of non-speech spikes caused by small gaps in speech segments is eliminated.

[0080] Furthermore, in the embodiments of this application, before companding the speech quality ratio of the current frame according to a set method to obtain the distortion of the current frame in step S130 above, a regression smoothing process is added to the speech quality ratio of the current frame to obtain the speech quality ratio of the current frame after smoothing.

[0081] It should be noted that the speech quality ratio of the current frame to be subjected to regression smoothing in this step can be calculated as RatioP, which is the signal energy after post-filtering and before post-filtering. i In addition, the speech quality ratio of the current frame can be calculated using the signal energy after post-filtering without smoothing and the signal energy before post-filtering without smoothing.

[0082] For ease of explanation, the following text uses the signal energy after post-filtering with smoothing and the signal energy before post-filtering with smoothing to calculate the speech quality ratio RatioP of the current frame. i Let's take an example to illustrate.

[0083] When performing regression smoothing on the speech quality ratio of the current frame, this embodiment provides a fast-rise, slow-fall smoothing strategy. That is, if the speech quality of the current frame is better than the previous frame, the speech quality ratio is updated quickly; if the speech quality of the current frame is worse than the previous frame, the speech quality ratio is updated slowly. Specific implementation methods may include:

[0084] S1. Calculate the difference Dif between the speech quality ratio of the current frame and the speech quality ratio after smoothing in the previous frame.

[0085] Dif = RatioP i -RatioPsGsc i-1

[0086] RatioPsGsc i-1 This indicates the audio quality ratio after smoothing the previous frame.

[0087] S2. If the difference Dif is greater than 0, then the speech quality ratio of the current frame is used as the speech quality ratio after smoothing the current frame.

[0088] S3. If the difference Dif is not greater than 0, the difference is multiplied by the first set weight, and then added to the smoothed speech quality ratio of the previous frame. The result is used as the smoothed speech quality ratio of the current frame.

[0089] The first set weight is less than 1, for example, it can be 0.08.

[0090]

[0091] Among them, RatioPsGsc i This indicates the speech quality ratio after smoothing the current frame.

[0092] As shown above, if the difference Dif between the speech quality ratio of the current frame and the smoothed speech quality ratio of the previous frame is greater than 0, then the smoothed speech quality ratio of the current frame is quickly updated to the speech quality ratio RatioP of the current frame. i If the difference Dif between the speech quality ratio of the current frame and the smoothed speech quality ratio of the previous frame is not greater than 0, then the difference is multiplied by a weight less than 1 and added to the smoothed speech quality ratio of the previous frame. The result is used as the smoothed speech quality ratio of the current frame, thus achieving slow updating of the speech quality ratio.

[0093] By performing regression smoothing on the speech quality ratio of the current frame, it can be ensured that signal frames with lower energy in the middle of a continuous sentence will not be mistaken for non-speech signals, thus eliminating the phenomenon of non-speech spikes caused by small gaps in speech segments.

[0094] Based on this, step S130 may specifically include:

[0095] The distortion of the current frame is obtained by companding the audio quality ratio of the smoothed current frame according to the set method.

[0096] In some embodiments of this application, the process of companding the speech quality ratio of the current frame according to a set method to obtain the distortion of the current frame in step S130 is described.

[0097] It should be noted that in this embodiment, the audio quality ratio of the current frame without smoothing can be companded, or the audio quality ratio after smoothing can be companded. This embodiment uses the companding of the audio quality ratio after smoothing as an example. For ease of description, the "audio quality ratio" mentioned below in this embodiment can be understood as "audio quality ratio after smoothing".

[0098] In this embodiment, a functional relationship between the speech quality ratio and the distortion can be preset. Based on this functional relationship, the distortion of the current frame corresponding to the speech quality ratio of the current frame can be calculated.

[0099] The functional relationship can include at least the following: when the speech quality ratio is less than a first value, the corresponding distortion value is 0; when the speech quality ratio is greater than the first value, the distortion is positively correlated with the speech quality ratio. The first value is a number greater than 0 and less than 0.5.

[0100] In this embodiment, if the speech quality ratio is less than the first value, it means that the speech quality ratio is within an acceptable range and will not affect subsequent data processing, such as subsequent speech recognition tasks. Therefore, the distortion can be set to 0 or a very small value.

[0101] The greater the distortion, the more severe the signal distortion.

[0102] Based on this, when the speech quality ratio is greater than the first value, the overall distortion is positively correlated with the speech quality ratio. Of course, according to the user's actual needs, the interval between the speech quality ratio being greater than the first value and less than 1 can be further divided into multiple intervals, and the rate of change of distortion with the speech quality ratio can be controlled by setting functional relationships in different intervals.

[0103] Referring to the following formula, this embodiment provides a functional relationship between an optional distortion degree 'a' and the speech quality ratio RatioPfGsc:

[0104]

[0105] The speech quality ratio is divided into three intervals from 0 to 1: 0-0.3, 0.3-0.5, and 0.5-1.

[0106] Furthermore, the distortion 'a' can be limited to the range of 0-1, resulting in, for example... Figure 3 The graph shown.

[0107] Figure 3 The horizontal axis represents the speech quality ratio, and the vertical axis represents the distortion degree. Figure 3 It can be seen that when the speech quality ratio is less than or equal to 0.3, the distortion value is 0.

[0108] In the range of 0.3-0.5, the ratio of distortion to speech quality exhibits the first slope;

[0109] In the 0.5-1 range, the distortion-to-speech-quality ratio exhibits a second slope, which is greater than the first slope. This indicates that as the speech-quality ratio increases, the rate of change of distortion also increases.

[0110] Of course, the above Figure 3 This example only illustrates one possible functional relationship between speech quality ratio and distortion. Users can also select other functional relationships between the two according to their actual needs.

[0111] The directional microphone array speech distortion detection scheme provided in the above embodiments of this application can perform distortion detection on the post-filter signal output by the post-filter in the directional microphone array, and obtain the distortion degree of each frame of the output post-filter signal.

[0112] Based on this, the embodiments of this application further provide a method for restoring speech distortion in a directional microphone array, which can restore the distortion of the post-filtered signal output by the post-filter. The distortion restoration process can be referred to... Figure 4The process shown includes the following steps:

[0113] Step S200: Obtain the distortion of the post-filter signal of the current frame output by the calculated post-filter.

[0114] Specifically, based on the various schemes described in the foregoing embodiments, the distortion 'a' of the post-filtered signal of the current frame output by the post-filter in the directional microphone array can be calculated.

[0115] Step S210: Based on the distortion, the current frame frequency domain signal input to the post-filter is mixed with the post-filter signal to obtain a mixed frequency domain signal.

[0116] Specifically, the distortion can be used as the mixing weight of the current frame frequency domain signal input to the post-filter, and then the current frame frequency domain signal can be mixed with the post-filter signal to obtain a mixed frequency domain signal.

[0117] Specifically, the mixed frequency domain signal can be represented as:

[0118] Y[k]=Pf_output(k)*(1.0-a)+Gsc_b0(k)*a

[0119] Where Y[k] represents the mixed frequency domain signal, Pf_output(k) represents the post-filter signal, Gsc_b0(k) represents the current frame frequency domain signal input to the post-filter, and k represents the frequency index. For each frequency point in a frame of signal, the distortion is the same.

[0120] As can be seen from the above formula, if the post-filtered signal of the current frame is completely undistorted, that is, the distortion degree 'a' is 0, then the mixed frequency domain signal is the post-filtered signal of the current frame; conversely, if the post-filtered signal of the current frame is completely distorted, that is, the distortion degree 'a' is 1, then the mixed frequency domain signal is the frequency domain signal of the current frame before the post-filter processing, indicating that the post-filtered signal is completely unusable at this time.

[0121] Step S220: Perform inverse Fourier transform on the mixed frequency domain signal to obtain a time-domain speech signal output.

[0122] Specifically, after the above-mentioned mixed frequency domain signal undergoes inverse Fourier transform (IFFT), a time-domain speech signal is obtained as the output:

[0123] output = IFFT(Y)

[0124] In some embodiments of this application, a directional microphone array voice distortion repair system is further provided, referring to... Figure 5 As shown, the system may include:

[0125] A fixed beamforming unit is used to extract speech from several directions using a fixed beamforming algorithm.

[0126] Specifically, the fixed beamforming unit can employ fixed beamforming algorithms such as Fixedbeamforming to extract speech from several directions: b0 to bn, based on the speech signal collected by the directional microphone array.

[0127] An adaptive beamforming unit is used to enhance the beam pointing of speech extracted from several directions by a fixed beamforming unit using an adaptive beamforming algorithm, thereby obtaining the frequency domain signal in the target direction and the frequency domain signal in other directions.

[0128] Specifically, the adaptive beamforming unit can use the generalized sidelobe cancellation (GSC) algorithm or other adaptive beamforming algorithms to enhance the beam pointing of the speech extracted from several directions by the fixed beamforming unit, thereby obtaining the frequency domain signal Target_b0 in the target direction and the frequency domain signals Ref_b1 to bn in other directions.

[0129] The post-filter is used to perform post-filtering processing on the signal output by the adaptive beamforming unit to obtain the post-filtered signal pf_output.

[0130] The distortion detection unit is used to calculate the distortion degree 'a' of the post-filtered signal pf_output of the current frame output by the post-filter using the distortion detection method described in the foregoing embodiments.

[0131] Specifically, the distortion a is calculated based on the frequency domain signal Target_b0 in the target direction and the post-filtered signal pf_output.

[0132] The distortion repair unit is used to mix the frequency domain signal Target_b0 in the target direction with the post-filtered signal pf_output based on the distortion degree a, to obtain a mixed frequency domain signal Y, and to perform an inverse Fourier transform on the mixed frequency domain signal Y to obtain a time domain speech signal output.

[0133] The following describes the directional microphone array speech distortion detection device provided in the embodiments of this application. The directional microphone array speech distortion detection device described below can be referred to in correspondence with the directional microphone array speech distortion detection method described above.

[0134] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a directional microphone array speech distortion detection device disclosed in an embodiment of this application.

[0135] like Figure 6 As shown, the device may include:

[0136] The first energy calculation unit 11 is used to calculate the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter.

[0137] The second energy calculation unit 12 is used to calculate the energy of the signal after post-filtering based on the post-filtered signal output by the post-filter.

[0138] The speech quality ratio calculation unit 13 is used to calculate the speech quality ratio of the current frame based on the signal energy after the post-filtering process and the signal energy before the post-filtering process.

[0139] The distortion calculation unit 14 is used to compress and expand the speech quality ratio of the current frame according to a set method to obtain the distortion of the current frame. The distortion is used to measure the distortion of the post-filtered signal of the current frame.

[0140] Optionally, the process by which the first energy calculation unit calculates the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter may include:

[0141] The energy of each frequency point in the mid-frequency band is calculated in the current frame frequency domain signal input to the post-filter and used as the signal energy before post-filtering.

[0142] Optionally, the process by which the second energy calculation unit calculates the energy of the post-filtered signal based on the post-filtered signal output by the post-filter may include:

[0143] The sum of the energy of each frequency point within the set mid-frequency band in the post-filter output signal is calculated as the signal energy after post-filtering.

[0144] Optionally, the apparatus of this application may further include:

[0145] An energy smoothing unit is used to perform regression smoothing on the signal energy before post-filtering and the signal energy after post-filtering before calculating the speech quality ratio of the current frame. Based on this, the process by which the speech quality ratio calculation unit calculates the speech quality ratio of the current frame based on the signal energy after post-filtering and the signal energy before post-filtering can include:

[0146] The speech quality ratio of the current frame is calculated based on the signal energy after post-filtering and the signal energy before post-filtering.

[0147] Optionally, the apparatus of this application may further include:

[0148] The speech quality ratio smoothing processing unit is used to perform regression smoothing processing on the speech quality ratio of the current frame before companding the speech quality ratio of the current frame according to a set method to obtain the distortion of the current frame, so as to obtain the speech quality ratio of the current frame after smoothing. Based on this, the process by which the distortion calculation unit compandspanns the speech quality ratio of the current frame according to the set method to obtain the distortion of the current frame may include:

[0149] The distortion of the current frame is obtained by companding the speech quality ratio after smoothing the current frame according to a set method.

[0150] Optionally, the process by which the above-mentioned speech quality ratio smoothing processing unit performs regression smoothing processing on the speech quality ratio of the current frame may include:

[0151] Calculate the difference between the speech quality ratio of the current frame and the speech quality ratio after smoothing in the previous frame;

[0152] If the difference is greater than 0, then the speech quality ratio of the current frame is used as the speech quality ratio after smoothing the current frame.

[0153] If the difference is not greater than 0, the difference is multiplied by the first set weight, and then added to the smoothed speech quality ratio of the previous frame. The result is used as the smoothed speech quality ratio of the current frame, where the first set weight is less than 1.

[0154] Optionally, the process by which the distortion calculation unit compresses and expands the speech quality ratio of the current frame according to a set method to obtain the distortion of the current frame may include:

[0155] Calculate the distortion of the current frame corresponding to the speech quality ratio of the current frame according to the set functional relationship between speech quality ratio and distortion.

[0156] The functional relationship includes at least the following: when the speech quality ratio is less than a first value, the corresponding distortion value is 0; when the speech quality ratio is greater than the first value, the distortion is positively correlated with the speech quality ratio; and the first value is a number greater than 0 and less than 0.5.

[0157] The following describes the directional microphone array speech distortion repair device provided in the embodiments of this application. The directional microphone array speech distortion repair device described below can be referred to in correspondence with the directional microphone array speech distortion repair method described above.

[0158] See Figure 7 , Figure 7 This is a schematic diagram of a directional microphone array voice distortion repair device disclosed in an embodiment of this application.

[0159] like Figure 7 As shown, the device may include:

[0160] The distortion acquisition unit 21 is used to acquire the distortion of the post-filter signal of the current frame output by the post-filter obtained by the distortion detection method described in the foregoing embodiments;

[0161] The signal mixing unit 22 is used to mix the current frame frequency domain signal input to the post-filter with the post-filter signal based on the distortion level to obtain a mixed frequency domain signal;

[0162] The signal transformation unit 23 is used to perform inverse Fourier transform on the mixed frequency domain signal to obtain a time domain speech signal output.

[0163] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0164] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0165] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting speech distortion using a directional microphone array, characterized in that, A method for detecting distortion in the post-filtered signal output from a post-filter in a directional microphone array includes: Calculate the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter; Based on the post-filtered signal output by the post-filter, calculate the signal energy after post-filtering. The speech quality ratio of the current frame is calculated based on the signal energy after post-filtering and the signal energy before post-filtering. Calculate the difference between the speech quality ratio of the current frame and the speech quality ratio after smoothing of the previous frame; if the difference is greater than 0, then use the speech quality ratio of the current frame as the speech quality ratio after smoothing of the current frame; if the difference is not greater than 0, then multiply the difference by a first set weight, and then add it to the speech quality ratio after smoothing of the previous frame, and use the result as the speech quality ratio after smoothing of the current frame, where the first set weight is less than 1. Based on the set functional relationship between the smoothed speech quality ratio and the distortion, calculate the distortion of the current frame corresponding to the smoothed speech quality ratio of the current frame. The functional relationship includes at least the following: when the smoothed speech quality ratio is less than a first value, the distortion of the current frame is 0; when the smoothed speech quality ratio is greater than the first value, the distortion of the current frame is positively correlated with the smoothed speech quality ratio; the first value is a number greater than 0 and less than 0.5; wherein the interval where the smoothed speech quality ratio is greater than the first value and less than 1 is divided into multiple intervals, and the rate of change of the distortion with the speech quality ratio is controlled by setting the functional relationship in different intervals; the distortion is used to measure the distortion of the post-filtered signal of the current frame.

2. The method according to claim 1, characterized in that, The calculation of the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter includes: The energy of each frequency point in the mid-frequency band is calculated in the current frame frequency domain signal input to the post-filter and used as the signal energy before post-filtering.

3. The method according to claim 1, characterized in that, The calculation of the signal energy after post-filtering based on the post-filter output signal includes: The sum of the energy of each frequency point within the set mid-frequency band in the post-filter output signal is calculated as the signal energy after post-filtering.

4. The method according to claim 1, characterized in that, Before calculating the speech quality ratio for the current frame, the following is also included: Regression smoothing is performed on the signal energy before and after the post-filtering process, respectively. The step of calculating the speech quality ratio of the current frame based on the signal energy after post-filtering and the signal energy before post-filtering includes: The speech quality ratio of the current frame is calculated based on the signal energy after post-filtering and the signal energy before post-filtering.

5. A method for restoring speech distortion using a directional microphone array, characterized in that, Methods for distortion correction of the post-filtered signal output from a post-filter in a directional microphone array include: Obtain the distortion of the post-filter signal of the current frame output by the method of any one of claims 1-4; Based on the distortion, the current frame frequency domain signal input to the post-filter is mixed with the post-filter signal to obtain a mixed frequency domain signal. The mixed frequency domain signal is subjected to inverse Fourier transform to obtain the time domain speech signal output.

6. A directional microphone array speech distortion detection device, characterized in that, The device for detecting distortion in the post-filtered signal output from a post-filter in a directional microphone array includes: The first energy calculation unit is used to calculate the signal energy before post-filtering based on the current frame frequency domain signal input to the post-filter. The second energy calculation unit is used to calculate the energy of the signal after post-filtering based on the post-filtered signal output by the post-filter. The speech quality ratio calculation unit is used to calculate the speech quality ratio of the current frame based on the signal energy after the post-filtering process and the signal energy before the post-filtering process. The speech quality ratio smoothing processing unit is used to calculate the difference between the speech quality ratio of the current frame and the smoothed speech quality ratio of the previous frame; if the difference is greater than 0, the speech quality ratio of the current frame is used as the smoothed speech quality ratio of the current frame; if the difference is not greater than 0, the difference is multiplied by a first preset weight, and then added to the smoothed speech quality ratio of the previous frame, and the result is used as the smoothed speech quality ratio of the current frame, wherein the first preset weight is less than 1. The distortion calculation unit is used to calculate the distortion of the current frame corresponding to the smoothed speech quality ratio of the current frame according to the set functional relationship between the smoothed speech quality ratio and the distortion. The functional relationship includes at least the following: when the smoothed speech quality ratio is less than a first value, the distortion of the current frame is 0; when the smoothed speech quality ratio is greater than the first value, the distortion of the current frame is positively correlated with the smoothed speech quality ratio; the first value is a number greater than 0 and less than 0.5; wherein the interval where the smoothed speech quality ratio is greater than the first value and less than 1 is divided into multiple intervals, and the rate of change of the distortion with the speech quality ratio is controlled by setting the functional relationship in different intervals; the distortion is used to measure the distortion of the post-filtered signal of the current frame.

7. A directional microphone array speech distortion repair device, characterized in that, The device is used to repair distortion in the post-filtered signal output from a post-filter in a directional microphone array. The distortion acquisition unit is used to acquire the distortion of the post-filtered signal of the current frame output by the post-filter obtained by the method of any one of claims 1-4. A signal mixing unit is used to mix the current frame frequency domain signal input to the post-filter with the post-filter signal based on the distortion level to obtain a mixed frequency domain signal; The signal transformation unit is used to perform inverse Fourier transform on the mixed frequency domain signal to obtain a time-domain speech signal output.

8. A directional microphone array speech distortion repair system, characterized in that, include: A fixed beamforming unit is used to extract speech from several directions using a fixed beamforming algorithm; An adaptive beamforming unit is used to enhance the beam pointing of speech extracted from several directions by a fixed beamforming unit using an adaptive beamforming algorithm, thereby obtaining the frequency domain signal of the target direction and the frequency domain signal of other directions. A post-filter is used to perform post-filtering processing on the signal output by the adaptive beamforming unit to obtain a post-filtered signal. The distortion detection unit is used to calculate the distortion of the post-filtered signal of the current frame output by the post-filter using the method of any one of claims 1-4. The distortion repair unit is used to mix the frequency domain signal in the target direction with the post-filtered signal based on the distortion degree to obtain a mixed frequency domain signal; and to perform an inverse Fourier transform on the mixed frequency domain signal to obtain a time-domain speech signal output.

9. The system according to claim 8, characterized in that, The adaptive beamforming algorithm is the Generalized Sidelobe Cancellation (GSC) algorithm.

Images