Method, apparatus and storage medium for processing audio signal

Abstract

The application discloses a processing method, device and equipment of an audio signal and a storage medium, the method comprises the following steps: determining a first covariance matrix of a pre-measured echo according to an excitation signal played by a loudspeaker in a quiet state; then obtaining a plurality of first audio signals containing the echo collected by a plurality of microphones and a reference signal transmitted in the loudspeaker; performing adaptive filtering processing on the plurality of first audio signals according to the reference signal to obtain a plurality of second audio signals after the adaptive filtering processing; performing beamforming processing on the plurality of second audio signals according to an inverse matrix of the first covariance matrix of the pre-measured echo to obtain a third audio signal after the beamforming processing; and performing noise elimination processing on the third audio signal to obtain a target audio signal after the noise elimination. According to the embodiment of the application, the echo residue can be effectively suppressed under the premise of ensuring that the distortion degree of the target voice is small.

Description

Technical Field

[0001] This application relates to the fields of signal processing and voice interaction technology, and in particular to a method, apparatus, device and storage medium for processing audio signals. Background Technology

[0002] Currently, acoustic devices with playback and acquisition functions are increasingly widely used. However, the echoes generated by the device itself can contaminate the target speech input by the target speaker. Therefore, echo cancellation algorithms are needed to cancel the echoes in the acquired signals and remove the signals played by the device itself.

[0003] Echo cancellation first involves audio acquisition and digital processing from the speaker's audio channel to obtain noise samples. Then, audio signals and noise are acquired through a microphone, and audio processing is performed by a DSP (Digital Signal Processing) processor to remove noise from the audio signals acquired by the microphone, thereby obtaining clean user-input target speech. This clean speech is then transmitted to relevant downstream systems for speech recognition and other functions.

[0004] Existing echo cancellation algorithms typically employ adaptive filtering and post-nonlinear filtering, such as the WebRTC architecture source code (Google's open-source technology for real-time video and audio communication) and the Speex open-source code. However, existing echo cancellation algorithms often result in significant echo residue and poor echo cancellation performance when the environment has high reverberation or significant nonlinearity of the speakers and microphones. Summary of the Invention

[0005] This application provides an audio signal processing method, apparatus, device, and storage medium. The audio signal processing method can effectively suppress echo remnants while ensuring that the target speech distortion is low.

[0006] In a first aspect, to achieve the above objective, embodiments of this application provide an audio signal processing method applied to an audio interactive device, the audio interactive device including multiple microphones and speakers, characterized in that the audio signal processing method includes:

[0007] Based on the excitation signal played by the loudspeaker in a quiet state, determine the first covariance matrix of the pre-measured echo;

[0008] Acquire multiple first audio signals containing echoes collected by the multiple microphones, and a reference signal transmitted within the speaker;

[0009] Based on the reference signal, multiple first audio signals are adaptively filtered to obtain multiple second audio signals after adaptive filtering.

[0010] Based on the inverse of the first covariance matrix, beamforming is performed on multiple second audio signals to obtain a beamformed third audio signal.

[0011] The third audio signal is subjected to noise cancellation processing to obtain the target audio signal after noise cancellation.

[0012] Furthermore, the step of adaptively filtering the plurality of first audio signals according to the reference signal to obtain a plurality of adaptively filtered second audio signals includes:

[0013] The echo estimate of the target microphone channel is determined based on the reference signal, and the echo estimate is used as the reference signal for other microphone channels to determine the echo estimate of other microphone channels.

[0014] Based on the echo estimate of each microphone channel and the first audio signal of the corresponding microphone channel, multiple echo-cancelled second audio signals are obtained;

[0015] The target microphone can be any one of the plurality of microphones.

[0016] Furthermore, the step of obtaining multiple echo-cancelled second audio signals based on the echo estimate of each microphone channel and the first audio signal of the corresponding microphone channel includes:

[0017] Subtract the echo estimate of the corresponding microphone channel from the first audio signal of each microphone channel to obtain multiple echo-cancelled second audio signals.

[0018] Furthermore, the step of determining the first covariance matrix of the pre-measured echo based on the excitation signal played by the speaker in a quiet state includes:

[0019] Obtain the operating time of the audio interaction device;

[0020] When the working time meets the preset time condition, the ambient sound is collected through the microphone, and it is detected whether the decibel value of the ambient sound is lower than the preset decibel value.

[0021] When the ambient sound level is lower than a preset decibel level, the speaker is controlled to play an excitation signal; the excitation signal is a sound signal covering the entire frequency band.

[0022] Calculate and determine the first covariance matrix of the sound signal.

[0023] Furthermore, when the azimuth angle of the target audio signal propagation is determined to be θ, the step of performing beamforming processing on the multiple second audio signals based on the inverse of the first covariance matrix includes:

[0024] The first weighting coefficients of the minimum variance distortionless response algorithm are determined based on the azimuth angle θ, the inverse of the first covariance matrix, and the first weighting coefficient formula.

[0025] The first minimum variance distortionless response algorithm is determined based on the first weight coefficient, and beamforming processing is performed on multiple second audio signals using the first minimum variance distortionless response algorithm.

[0026] The first formula for calculating the weighting coefficients is: ;

[0027] in, Let A be the inverse of the first covariance matrix, and let A be the steering vector determined based on the azimuth angle θ. , Indicates the number of frequency points. , Represents the natural constant Exponentiation Represents the imaginary unit. , This indicates that the target audio signal has propagated to the first... The time difference between the time of the target audio signal reaching the origin of the coordinate system and the time of the microphone. , It indicates the speed of sound.

[0028] Furthermore, when the azimuth angle of the target audio signal propagation cannot be determined, the step of performing beamforming processing on multiple second audio signals based on the inverse of the first covariance matrix includes:

[0029] The second weighting coefficients of the minimum variance distortionless response algorithm are determined based on the inverse matrix of the first covariance matrix, the second covariance matrices of the multiple second audio signals, and the second weighting coefficient formula.

[0030] The second minimum variance distortionless response algorithm is determined based on the second weight coefficient, and beamforming is performed on multiple second audio signals using the second minimum variance distortionless response algorithm.

[0031] The second formula for calculating the weighting coefficients is: ;

[0032] in, This represents the covariance matrix of the second audio signal. express The identity matrix, Represents the trace of a matrix. And there are A vector of n elements.

[0033] Furthermore, the step of performing noise cancellation processing on the third audio signal to obtain the noise-cancelled target audio signal includes:

[0034] The noise in the third audio signal is calculated based on the third audio signal and the noise calculation formula.

[0035] Based on the noise in the third audio signal, Wiener filtering is performed on the third audio signal to obtain the target audio signal after noise removal;

[0036] The noise calculation formula is as follows: , As a leakage factor, For frame index, This refers to the third audio signal.

[0037] In a second aspect, in order to solve the same technical problem, embodiments of the present invention also provide an audio signal processing device applied to an audio interactive device, the audio interactive device including multiple microphones and speakers, characterized in that the audio signal processing device includes: an echo determination module, a signal acquisition module, a first processing module, a second processing module, and an echo cancellation module;

[0038] The echo determination module is used to determine a first covariance matrix of the pre-measured echo based on the excitation signal played by the speaker in a quiet state.

[0039] The signal acquisition module is used to acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted in the speaker;

[0040] The first processing module is configured to perform adaptive filtering on a plurality of first audio signals according to the reference signal to obtain a plurality of adaptively filtered second audio signals;

[0041] The second processing module is used to perform beamforming processing on multiple second audio signals based on the inverse matrix of the first covariance matrix to obtain a beamformed third audio signal.

[0042] The echo cancellation module is used to perform noise cancellation processing on the third audio signal to obtain the target audio signal after noise cancellation.

[0043] In a third aspect, in order to solve the same technical problem, embodiments of the present invention also provide an audio interaction device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the memory being coupled to the processor, and the processor executing the computer program implementing the audio signal processing method as described in any of the preceding claims.

[0044] In a fourth aspect, in order to solve the same technical problem, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the audio signal processing method described in any of the above claims.

[0045] The audio signal processing method, apparatus, device, and storage medium provided in this application embodiment can enhance the target audio signal and reduce the distortion of the target speech by performing beamforming on the echo-cancelled audio signal and noise cancellation on the beamformed audio signal, and can also effectively suppress echo residue. Attached Figure Description

[0046] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0047] Figure 1 A flowchart illustrating the audio signal processing method provided in this application embodiment;

[0048] Figure 2 A schematic diagram of the multi-microphone channel adaptive filtering process provided in an embodiment of this application;

[0049] Figure 3 Another schematic flowchart illustrating the audio signal processing method provided in this application embodiment;

[0050] Figure 4 A schematic diagram of the structure of the audio signal processing apparatus provided in the embodiments of this application;

[0051] Figure 5 This is a schematic diagram of the structure of the second processing module provided in an embodiment of this application;

[0052] Figure 6 A schematic diagram of the structure of an audio interaction device provided in an embodiment of this application;

[0053] Figure 7 This is another structural schematic diagram of the audio interaction device provided in the embodiments of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0055] The audio signal processing method provided in this invention is mainly used in audio interactive devices. Since audio interactive devices are equipped with microphones and speakers, this application primarily targets audio interactive devices with multiple microphones. Therefore, please refer to... Figure 1 , Figure 1 This is a flowchart illustrating an audio signal processing method provided in an embodiment of the present application. In this embodiment of the present invention, the audio signal processing method includes steps S101 to S105.

[0056] Step S101: Determine the first covariance matrix of the pre-measured echo based on the excitation signal played by the speaker in a quiet state.

[0057] In one implementation, the covariance matrix of the pre-measured echo is the covariance matrix of the echo of the audio interaction device itself. Specifically, the method for determining the first covariance matrix of the pre-measured echo based on the excitation signal played by the speaker in a quiet state is as follows: obtain the working time of the audio interaction device; when the working time meets the preset time condition, collect ambient sound through the microphone and detect whether the decibel value of the ambient sound is lower than the preset decibel value; when the decibel value of the ambient sound is lower than the preset decibel value, control the speaker to play the excitation signal, and calculate and determine the first covariance matrix of the sound signal.

[0058] It should be noted that existing technologies require estimating the covariance matrix of the audio interaction device's own echo before performing beamforming. However, this application determines in advance whether the covariance matrix of the audio interaction device's own echo needs to be re-estimated based on the operating time of the audio interaction device. This avoids the waste of a large amount of computing resources caused by needing to estimate the covariance matrix in real time every time beamforming is performed. It also ensures that beamforming produces a zero limit on the echo channel, thereby effectively suppressing echo residue.

[0059] In this embodiment, the preset time condition is used to determine whether the audio interaction device is powered on again. Therefore, the preset time condition should be set as short as possible. In this embodiment, the preset time condition is no more than 10 seconds, but the preset time condition can also be set to no more than 15 seconds, 30 seconds, or 1 minute, etc. The specific time condition can be determined according to the actual situation.

[0060] It should be noted that when the working time of the audio interaction device meets the preset time conditions, such as when the working time of the audio interaction device is detected to be 5 seconds, which does not exceed the preset time condition of 10 seconds, it can be determined that the audio interaction device is in a re-powered state.

[0061] Optionally, it can be determined whether the inverse of the first covariance matrix of the pre-measured echo needs to be re-estimated by directly detecting whether the audio interaction device is powered on again.

[0062] When the working time of the audio interaction device meets the preset time condition, or when the audio interaction device is detected to be powered on again, it is necessary to re-estimate the covariance matrix of the audio interaction device's own echo. Therefore, in order to ensure the accuracy of the re-estimated covariance matrix, this embodiment of the invention detects whether the current environment is a quiet environment, so as to ensure that the covariance matrix of the audio interaction device's own echo is estimated in a quiet environment, thereby avoiding the influence of noisy or uncontrollable noise environment on the process of estimating the covariance matrix.

[0063] Furthermore, in order to reduce the computational resources required for real-time estimation before each beamforming process, a preset time threshold can be used, and the timing of the timer can be monitored in real time. When the timer reaches the time threshold, the covariance matrix of the audio interaction device's echo is re-estimated.

[0064] As an optional embodiment, this embodiment collects ambient sound through the microphone of the audio interaction device and detects whether the decibel value of the ambient sound is lower than a preset decibel value. When the decibel value of the ambient sound is lower than the preset decibel value, it is determined that the current environment is a quiet environment. Optionally, the preset decibel value is set to 50 decibels. Specifically, a specific decibel value can also be set according to the actual situation, such as 40, 45, or 55 decibels.

[0065] In this embodiment, the excitation signal is a sound signal covering the entire frequency band, including white noise, maximum length sequence signal, speech, and other signals. In this embodiment of the invention, white noise is used as the excitation signal. White noise refers to noise with a power spectral density that is uniformly distributed throughout the entire frequency domain.

[0066] In an embodiment of the present invention, when the operating duration of the audio interaction device meets a preset time condition, or when it is determined that the audio interaction device is powered on again, or when the timer reaches a preset time threshold, a white noise signal covering the entire frequency band for a preset duration, such as 3 seconds, is played through the speaker of the audio interaction device. This white noise signal propagates through space to the M microphones of the audio interaction device, thereby obtaining the time-domain signal of the echo of the audio interaction device itself. , where m=1,2,...M, and t is the time index of the signal point.

[0067] Obtaining the time-domain signal Then, this time-domain signal The frequency domain signal is obtained by performing frame segmentation, windowing, and Fourier transform. ,in Here, k is the frame index, k=1,2,...K, where K is the number of frequency points. The covariance matrix of the audio interaction device's own echo is calculated using the following covariance matrix formula:

[0068]

[0069] in, Rnn(m,n,k) represents the element in the m-th row and n-th column of the Rnn(k) matrix, L is the total number of frames, m represents the m-th microphone, n represents the n-th microphone, and m,n=1,2,...M.

[0070] In an embodiment of the present invention, when the working duration of the audio interaction device does not meet the preset time condition, or when it is determined that the audio interaction device has not been powered on again, or when the timer time has not reached the preset time threshold, the first covariance matrix estimated last time is used again, which can get rid of the waste of a lot of computing resources caused by estimating the covariance matrix in real time every time.

[0071] In another implementation, the first covariance matrix of the echo can be pre-measured and determined by professional engineers during the production of the audio interactive device based on the excitation signal played by the speaker in a quiet state. This enables the audio interactive device to provide ordinary users with a pre-set, standard first covariance matrix of the echo of the audio interactive device itself after production, allowing ordinary users to directly use it to make the audio interactive device work properly.

[0072] Step S102: Acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted within the speaker.

[0073] In this embodiment, the audio interaction device is configured to have M microphones and one speaker. Therefore, this embodiment is able to acquire the first audio signal containing echoes from the M microphone channels. and the reference signal transmitted within the speaker ,in, This represents the signal received by the m-th microphone at time t, where m = 1, 2, ..., M, and t is the time index of the signal point. This represents the signal received by the loudspeaker at time t.

[0074] Step S103: Perform adaptive filtering on multiple first audio signals based on the reference signal to obtain multiple adaptively filtered second audio signals.

[0075] In this embodiment, before performing adaptive filtering, it is necessary to process the first audio signal in the time domain. and reference signal The process involves framing, windowing, and Fourier transform to obtain the first audio signal in the frequency domain. and reference signal ,in, Indicates the m-th channel, the... The frame, the signal at the k-th frequency point, Indicates the first The frame represents the signal at the k-th frequency point, where k represents the frequency point number, k = 1, 2, ..., K, and K is the total number of frequency points.

[0076] After obtaining the frequency domain signal, based on the reference signal For each first audio signal Adaptive filtering is performed to obtain M second audio signals after adaptive filtering.

[0077] As an optional embodiment of the present invention, step S103 includes: determining the echo estimate of the target microphone channel according to the reference signal, and using the echo estimate as the reference signal for other microphone channels to determine the echo estimate of other microphone channels, and obtaining a plurality of echo-cancelled second audio signals based on the echo estimate of each microphone channel and the first audio signal of the corresponding microphone channel.

[0078] It should be noted that the target microphone is any one of the M microphones in the audio interaction device. Preferably, the target microphone is the first microphone in the audio interaction device that undergoes adaptive filtering.

[0079] In this embodiment, the step of obtaining multiple echo-cancelled second audio signals based on the echo estimation value of each microphone channel and the first audio signal of the corresponding microphone channel is as follows: subtract the echo estimation value of the corresponding microphone channel from the first audio signal of each microphone channel to obtain multiple echo-cancelled second audio signals.

[0080] Since the echo estimate is equivalent to the echo in the corresponding microphone channel, subtracting the echo estimate of the corresponding microphone channel from the acquired first audio signal containing the echo can yield the second audio signal after echo cancellation.

[0081] In this embodiment, the target microphone channel is adaptively filtered using the NLMS (Normalized Least Mean Square) algorithm. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the multi-microphone channel adaptive filtering process provided in the embodiments of this application, as shown below. Figure 2 As shown, the first audio signal in the time domain and reference signal Converted to the first audio signal in the frequency domain and reference signal Then, the reference signal transmitted within the speaker 10 is filtered by filter 31. Echo estimation is performed to obtain the echo estimate of the target microphone channel 21. Then, the echo estimate of the target microphone channel 21 is... As a reference signal for echo estimation of the other microphone channels 22-2M, the echo estimation value of the corresponding microphone channel is subtracted from the first audio signal of each microphone channel to obtain the second audio signal after adaptive filtering of M microphone channels. .

[0082] It should be noted that the echo estimation of the target microphone channel is performed in the following way:

[0083] ;

[0084] in, It is a reference signal Historical buffer value, The filter coefficients are the preset NLMS filter coefficients. Obtained through the following methods:

[0085] ;

[0086] In this embodiment, It is the order of NLMS.

[0087] After obtaining the echo estimate of the target microphone channel Then, the echo-free second audio signal is obtained in the following way. :

[0088] ;

[0089] It should be noted that after each frame of adaptive filtering is completed, the NLMS filter coefficients need to be updated to determine the signal for the next frame.

[0090] ;

[0091] In this embodiment, It is the step size adjustment factor. This indicates the search for conjugate.

[0092] For further information, please refer to [link / reference]. Figure 2After completing the adaptive filtering process for the target microphone channel, echo estimation is performed on the Mth microphone channel in the following manner:

[0093] ;

[0094] in, It is a reference signal Historical buffer value, These are the filter coefficients of the preset NLMS filter.

[0095] Following the same adaptive filtering method applied to the target microphone channel, adaptive filtering is sequentially applied to all other microphone channels to obtain the second audio signal after adaptive filtering of M microphone channels. .

[0096] By using the method provided in this embodiment of the invention, the echo estimate of the target microphone channel can be used as a reference signal for other microphone channels, thereby reducing the filter order when performing adaptive filtering on other microphone channels, thus reducing memory usage and wasting computing resources.

[0097] It should be noted that the algorithm used in the adaptive filtering process can also be frequency domain algorithms such as LMS, RLS, and Kalman filter, or time domain algorithms such as NLMS, LMS, RLS, and Kalman filter that can perform adaptive filtering processing; there are no restrictions here.

[0098] Step S104: Based on the inverse of the first covariance matrix, beamforming is performed on multiple second audio signals to obtain a beamformed third audio signal.

[0099] In this embodiment, MVDR (Minimum variance distortionless response) is used as the beamforming algorithm. Therefore, the third audio signal after beamforming can be obtained according to the MVDR algorithm. The third audio signal is obtained in the following way:

[0100] ;

[0101] in, The second audio signal after adaptive filtering of M microphone channels. , These are the weight coefficients for the MVDR algorithm. To obtain the conjugate transpose.

[0102] In one embodiment of the present invention, when the azimuth angle of the target audio signal propagation is determined to be θ, step S104 includes: determining the first weight coefficients of the minimum variance distortionless response algorithm based on the azimuth angle θ, the inverse matrix of the first covariance matrix, and the first weight coefficient formula; determining the first minimum variance distortionless response algorithm based on the first weight coefficients; and performing beamforming processing on multiple second audio signals through the first minimum variance distortionless response algorithm to obtain a beamformed third audio signal.

[0103] It should be noted that the inverse of the first covariance matrix is ​​obtained by inverting the first covariance matrix; therefore, the inverse of the first covariance matrix is... .

[0104] In this embodiment, the first formula for calculating the weighting coefficients is:

[0105] ;

[0106] in, Let represent the inverse of the first covariance matrix, and A be the steering vector determined based on the azimuth angle θ. , Indicates the number of frequency points. Represents the natural constant Exponentiation, Represents the imaginary unit. , Indicates that the target audio signal has propagated to the first... The time difference between the arrival of the target audio signal at the origin and the arrival of the target audio signal at the microphone. , It indicates the speed of sound.

[0107] In another embodiment of the present invention, when the azimuth angle of the target audio signal propagation cannot be determined, step S104 further includes: determining the second weight coefficients of the minimum variance distortionless response algorithm based on the inverse matrix of the first covariance matrix, the second covariance matrix of the multiple second audio signals, and the second weight coefficient formula; determining the second minimum variance distortionless response algorithm based on the second weight coefficients; and performing beamforming processing on the multiple second audio signals using the second minimum variance distortionless response algorithm to obtain a beamformed third audio signal.

[0108] In this embodiment, the formula for the second weighting coefficient is:

[0109] ;

[0110] in, This represents the covariance matrix of the second audio signal. express The identity matrix, Represents the trace of a matrix. And there are A vector of n elements.

[0111] In this embodiment, the covariance matrix of the second audio signal is obtained in the following way:

[0112] ;

[0113] in, .

[0114] By using the beamforming technique mentioned above to weight and synthesize multiple second audio signals to form a single desired third audio signal, the target audio signal can be effectively enhanced while minimizing distortion of the target speech.

[0115] Step S105: Perform noise cancellation processing on the third audio signal to obtain the target audio signal after noise cancellation.

[0116] In this embodiment, step S105 specifically involves: calculating the noise in the third audio signal based on the third audio signal and the noise calculation formula; performing Wiener filtering on the third audio signal based on the noise in the third audio signal to obtain the target audio signal after noise removal.

[0117] The formula for calculating noise is:

[0118] ;

[0119] in, As a leakage factor, This is the third audio signal after beamforming.

[0120] In this embodiment, leakage factor It is calculated using the following formula:

[0121] ;

[0122] in, , ,

[0123] and, , ,

[0124] ,

[0125] ,

[0126] ,

[0127] ,

[0128] It should be noted that, It is the base learning rate of the noise, in this embodiment Take 0.6, This indicates taking the minimum of the two values.

[0129] After obtaining the noise in the third audio signal using the above method, Wiener filtering is performed on the third audio signal based on the noise obtained. Specifically, Wiener filtering is performed using the following formula:

[0130] ;

[0131] in, .

[0132] In this embodiment, the target audio signal in the frequency domain obtained after Wiener filtering is... Subsequently, the audio signal processing method provided in this embodiment of the invention further includes: processing the frequency domain signal... Windowing, inverse Fourier transform, and merging are performed to obtain the target audio signal in the time domain. .

[0133] For a preferred embodiment of the present invention, please refer to Figure 3 , Figure 3 Another schematic flowchart of the audio signal processing method provided in the embodiments of this application is shown below. Figure 3 As shown, the audio signal processing method provided in this embodiment of the invention includes steps S201 to S213;

[0134] Step S201: Obtain the working duration of the audio interaction device.

[0135] The duration of operation of the audio interactive device can be obtained by pre-setting a timer that starts counting when the audio interactive device is powered on again.

[0136] Step S202: When the working time meets the preset time condition, the ambient sound is collected through the microphone, and the decibel value of the ambient sound is detected as lower than the preset decibel value.

[0137] In this embodiment, a preset time condition is used to determine whether the audio interaction device has been re-powered. Therefore, by using the time data recorded by the preset timer and the preset time condition, it can be determined whether the audio interaction device has been re-powered.

[0138] It should be noted that if the working time does not meet the preset time condition, the process will proceed to step S205.

[0139] Step S203: When the ambient sound decibel value is lower than the preset decibel value, control the speaker to play a white noise signal covering the entire frequency band.

[0140] In this embodiment, when the ambient sound decibel level is lower than the preset 50 decibels, the speaker is controlled to play a 3-second white noise signal covering the entire frequency band.

[0141] Step S204: Calculate and determine the first covariance matrix of the white noise signal.

[0142] Step S205: Acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted within the speaker.

[0143] In this embodiment, the audio interaction device is equipped with M microphones and 1 speaker. Therefore, this embodiment can acquire the first audio signal of the M microphone channels and 1 reference signal, wherein the first audio signal contains echo.

[0144] Step S206: Determine the echo estimate of the target microphone channel based on the reference signal, and use the echo estimate as the reference signal for other microphone channels to determine the echo estimates of other microphone channels.

[0145] Step S207: Subtract the echo estimate of the corresponding microphone channel from the first audio signal of each microphone channel to obtain multiple echo-cancelled second audio signals.

[0146] The target microphone can be any one of multiple microphones.

[0147] By using the method provided in this embodiment of the invention, the echo estimate of the target microphone channel can be used as a reference signal for other microphone channels, thereby reducing the filter order when performing adaptive filtering on other microphone channels, thus reducing memory usage and wasting computing resources.

[0148] Step S208: When the azimuth angle of the target audio signal propagation is determined to be θ, the first weight coefficient of the minimum variance distortionless response algorithm is determined according to the azimuth angle θ, the inverse matrix of the first covariance matrix, and the first weight coefficient formula.

[0149] In this embodiment, the method for determining the first weight coefficient of the minimum variance distortionless response algorithm is the same as that in the above embodiment, which determines the content corresponding to the first weight coefficient of the minimum variance distortionless response algorithm, and will not be repeated here.

[0150] Step S209: Determine the first minimum variance distortionless response algorithm based on the first weight coefficient, and perform beamforming processing on multiple second audio signals using the first minimum variance distortionless response algorithm to obtain a beamformed third audio signal.

[0151] Step S210: When the azimuth angle of the target audio signal propagation cannot be determined, the second weight coefficients of the minimum variance distortionless response algorithm are determined based on the inverse matrix of the first covariance matrix, the second covariance matrix of multiple second audio signals, and the second weight coefficient formula.

[0152] In this embodiment, the method for determining the second weight coefficients of the minimum variance distortionless response algorithm is the same as that in the above embodiment, which determines the content corresponding to the second weight coefficients of the minimum variance distortionless response algorithm, and will not be repeated here.

[0153] Step S211: Determine the second minimum variance distortionless response algorithm based on the second weight coefficients, and perform beamforming processing on multiple second audio signals using the second minimum variance distortionless response algorithm to obtain a beamformed third audio signal.

[0154] Step S212: Calculate the noise in the third audio signal based on the third audio signal and the noise calculation formula.

[0155] In this embodiment, the method for calculating the noise in the third audio signal is the same as that in the above embodiment, and will not be repeated here.

[0156] Step S213: Based on the noise in the third audio signal, Wiener filtering is performed on the third audio signal to obtain the target audio signal after noise removal.

[0157] The noise is determined by analyzing the third audio signal after beamforming, which can then be removed using Wiener filtering. This further suppresses residual echoes, effectively suppressing residual echoes while ensuring minimal distortion of the target speech.

[0158] After obtaining the target audio signal in the frequency domain through Wiener filtering, the audio signal processing method provided in this embodiment of the invention further includes: windowing, inverse Fourier transform, and merging processing of the frequency domain signal to obtain the target audio signal in the time domain.

[0159] In summary, the embodiments of the present invention provide an audio signal processing method. This method includes determining a first covariance matrix for pre-measuring echoes based on an excitation signal played by a speaker in a quiet state; acquiring multiple first audio signals containing echoes collected by multiple microphones and a reference signal transmitted within the speaker; performing adaptive filtering on the multiple first audio signals based on the reference signal to obtain multiple adaptively filtered second audio signals; performing beamforming on the multiple second audio signals based on the inverse of the first covariance matrix to obtain a beamformed third audio signal; and performing noise cancellation on the third audio signal to obtain a noise-cancelled target audio signal. Using the embodiments of the present invention, echo remnants can be effectively suppressed while ensuring minimal distortion of the target speech.

[0160] Based on the method described in the above embodiments, this embodiment will be further described from the perspective of an audio signal processing device. The audio signal processing device can be implemented as a separate entity or integrated into an electronic device, such as a terminal, which may include a mobile phone, a tablet computer, etc.

[0161] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an audio signal processing device provided in an embodiment of this application. The audio signal processing device is mainly used in audio interactive devices, and the audio interactive device includes multiple microphones and speakers, such as... Figure 4 As shown, the audio signal processing device 400 provided in this embodiment of the invention includes an echo determination module 401, a signal acquisition module 402, a first processing module 403, a second processing module 404, and an echo cancellation module 405.

[0162] The echo determination module 401 is used to determine a first covariance matrix of the pre-measured echo based on the excitation signal played by the speaker in a quiet state.

[0163] In this embodiment, the echo determination module is specifically used to: obtain the working time of the audio interaction device; when the working time meets the preset time condition, collect ambient sound through the microphone and detect whether the decibel value of the ambient sound is lower than the preset decibel value; when the decibel value of the ambient sound is lower than the preset decibel value, control the speaker to play an excitation signal, the excitation signal being a sound signal covering the entire frequency band, and calculate and determine the first covariance matrix of the sound signal.

[0164] The signal acquisition module 402 is used to acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted in the speaker.

[0165] The first processing module 403 is used to perform adaptive filtering on multiple first audio signals according to a reference signal to obtain multiple adaptively filtered second audio signals.

[0166] In this embodiment, the first processing module 403 is specifically used to: determine the echo estimate of the target microphone channel based on the reference signal, and use the echo estimate as the reference signal for other microphone channels to determine the echo estimate of other microphone channels, and obtain multiple echo-cancelled second audio signals based on the echo estimate of each microphone channel and the first audio signal of the corresponding microphone channel.

[0167] The target microphone can be any one of multiple microphones.

[0168] It should be noted that the first processing module 403 is further used to: subtract the echo estimate of the corresponding microphone channel from the first audio signal of each microphone channel to obtain multiple echo-cancelled second audio signals.

[0169] In this embodiment, the adaptive filtering process for each microphone channel is described in the above method embodiment and will not be repeated here.

[0170] The second processing module 404 is used to perform beamforming processing on multiple third audio signals based on the inverse matrix of the first covariance matrix to obtain a beamformed fourth audio signal.

[0171] In this embodiment, the method for calculating the inverse matrix of the first covariance matrix of the pre-measured echo is the same as that mentioned in the above method embodiment, and will not be repeated here.

[0172] Optionally, the beamforming process in this embodiment can be implemented using the MVDR algorithm or by multi-channel distortionless Wiener filtering.

[0173] In this embodiment, please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of the second processing module provided in the embodiments of this application, as shown below. Figure 5 As shown, the second processing module 404 includes a first determining unit 4041, a first processing unit 4042, a second determining unit 4043, and a second processing unit 4044.

[0174] The first determining unit 4041 is used to determine the first weighting coefficient of the minimum variance distortionless response algorithm based on the azimuth angle θ, the inverse matrix of the first covariance matrix, and the first weighting coefficient formula when the azimuth angle of the target audio signal propagation is determined to be θ.

[0175] In this embodiment, the method for determining the first weight coefficient of the minimum variance distortionless response algorithm is the same as that described in the above method embodiment, and will not be repeated here.

[0176] The first processing unit 4042 is used to determine the first minimum variance distortionless response algorithm based on the first weight coefficient, and to perform beamforming processing on multiple second audio signals through the first minimum variance distortionless response algorithm.

[0177] The second determining unit 4043 is used to determine the second weighting coefficients of the minimum variance distortionless response algorithm based on the inverse matrix of the first covariance matrix, the second covariance matrix of multiple second audio signals, and the second weighting coefficient formula when the azimuth angle of the target audio signal propagation cannot be determined.

[0178] In this embodiment, the method for determining the second weight coefficients of the minimum variance distortionless response algorithm is the same as that described in the above method embodiment, and will not be repeated here.

[0179] The second processing unit 4044 is used to determine the second minimum variance distortionless response algorithm based on the second weight coefficients, and to perform beamforming processing on multiple second audio signals through the second minimum variance distortionless response algorithm.

[0180] The echo cancellation module 405 is used to perform noise cancellation processing on the third audio signal to obtain the target audio signal after noise cancellation.

[0181] In this embodiment, the echo cancellation module 405 is specifically used to: calculate the noise in the third audio signal based on the third audio signal and the noise calculation formula, and perform Wiener filtering on the third audio signal based on the noise in the third audio signal to obtain the target audio signal after noise cancellation.

[0182] It should be noted that the method for calculating noise is the same as that mentioned in the above method embodiments, and will not be repeated here.

[0183] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0184] Additionally, please see Figure 6 , Figure 6This is a schematic diagram of the structure of an audio interaction device provided in an embodiment of this application. The audio interaction device can be a mobile terminal such as a smartphone or tablet computer. Figure 6 As shown, the audio interaction device 600 includes a processor 601 and a memory 602. The processor 601 and the memory 602 are electrically connected.

[0185] The processor 601 is the control center of the audio interaction device 600. It connects various parts of the electronic device through various interfaces and lines. By running or loading the application program stored in the memory 602 and calling the data stored in the memory 602, it executes various functions of the audio interaction device 600 and processes data, thereby performing overall monitoring of the audio interaction device 600.

[0186] In this embodiment, the processor 601 in the audio interaction device 600 loads the instructions corresponding to the processes of one or more applications into the memory 602 according to the following steps, and the processor 601 runs the applications stored in the memory 602 to realize various functions:

[0187] The first covariance matrix of the pre-measured echo is determined based on the excitation signal played by the loudspeaker in a quiet state;

[0188] Acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted within the speaker;

[0189] Multiple first audio signals are adaptively filtered based on a reference signal to obtain multiple second audio signals after adaptive filtering.

[0190] Based on the inverse of the first covariance matrix, beamforming is performed on multiple second audio signals to obtain a beamformed third audio signal.

[0191] The third audio signal is subjected to noise cancellation processing to obtain the target audio signal after noise cancellation.

[0192] The audio interaction device 600 can implement the steps of any embodiment of the audio signal processing method provided in the embodiments of the present invention. Therefore, it can achieve the beneficial effects that any audio signal processing method provided in the embodiments of the present invention can achieve. For details, please refer to the previous embodiments, which will not be repeated here.

[0193] Please see Figure 7 , Figure 7 This is another structural schematic diagram of the audio interaction device provided in the embodiments of this application, such as... Figure 7 As shown, Figure 7A specific structural block diagram of an audio interaction device provided in an embodiment of the present invention is shown. This audio interaction device can be used to implement the audio signal processing method provided in the above embodiments. The audio interaction device 700 can be a mobile terminal such as a smartphone or a laptop computer.

[0194] RF circuit 710 is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals and vice versa, thereby enabling communication with communication networks or other devices. RF circuit 710 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity modules (SIM cards), memory, etc. RF circuit 710 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). The aforementioned wireless networks may use various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messages, and any other suitable communication protocols, including those that have not yet been developed.

[0195] The memory 720 can be used to store software programs and modules, such as the program instructions / modules corresponding to the audio signal processing method in the above embodiment. The processor 780 executes various functional applications and data processing by running the software programs and modules stored in the memory 720, that is, it realizes the following functions:

[0196] The first covariance matrix of the pre-measured echo is determined based on the excitation signal played by the loudspeaker in a quiet state;

[0197] Acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted within the speaker;

[0198] Multiple first audio signals are adaptively filtered based on a reference signal to obtain multiple second audio signals after adaptive filtering.

[0199] Based on the inverse of the first covariance matrix, beamforming is performed on multiple second audio signals to obtain a beamformed third audio signal.

[0200] The third audio signal is subjected to noise cancellation processing to obtain the target audio signal after noise cancellation.

[0201] Memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, memory 720 may further include memory remotely located relative to processor 780, which can be connected to audio interaction device 700 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0202] The input unit 730 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 730 may include a touch-sensitive surface 731 and other input devices 732. The touch-sensitive surface 731, also known as a touch display screen or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 731), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 731 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 780, and can receive and execute commands sent by the processor 780. In addition, the touch-sensitive surface 731 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 731, the input unit 730 may also include other input devices 732. Specifically, other input devices 732 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0203] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the audio interaction device 700. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit 740 may include a display panel 741, which may optionally be configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or similar display. Furthermore, a touch-sensitive surface 731 may cover the display panel 741. When the touch-sensitive surface 731 detects a touch operation on or near it, it transmits the information to the processor 780 to determine the type of touch event. Subsequently, the processor 780 provides corresponding visual output on the display panel 741 according to the type of touch event. Although in the figures, the touch-sensitive surface 731 and the display panel 741 are implemented as two separate components to achieve input and output functions, in some embodiments, the touch-sensitive surface 731 and the display panel 741 can be integrated to achieve input and output functions.

[0204] The audio interaction device 700 may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 741 according to the ambient light level, and the proximity sensor can generate an interruption when the flip is closed or shut down. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. As for other sensors that may be configured in the audio interaction device 700, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, they will not be described in detail here.

[0205] Audio circuitry 760, speaker 761, and microphone 762 provide an audio interface between the user and audio interaction device 700. Audio circuitry 760 converts received audio data into electrical signals, which are then transmitted to speaker 761, where they are converted into sound signals for output. Conversely, microphone 762 converts collected sound signals into electrical signals, which are received by audio circuitry 760, converted back into audio data, and then processed by processor 780 before being transmitted via RF circuitry 710 to, for example, another terminal, or output to memory 720 for further processing. Audio circuitry 760 may also include an earphone jack to facilitate communication between peripheral headphones and audio interaction device 700.

[0206] The audio interaction device 700, through a transmission module 770 (such as a Wi-Fi module), enables users to receive requests and send information, providing them with wireless broadband internet access. Although the transmission module 770 is shown in the figure, it is understood that it is not an essential component of the audio interaction device 700 and can be omitted as needed without altering the essence of the invention.

[0207] The processor 780 is the control center of the audio interaction device 700. It connects to various parts of the mobile phone via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 720, and by calling data stored in the memory 720, it performs various functions and processes data of the audio interaction device 700, thereby providing overall monitoring of the electronic device. Optionally, the processor 780 may include one or more processing cores; in some embodiments, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 780.

[0208] The audio interaction device 700 also includes a power supply 790 (such as a battery) that supplies power to the various components. In some embodiments, the power supply may be logically connected to the processor 780 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. The power supply 790 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0209] Although not shown, the audio interaction device 700 also includes a camera (such as a front-facing camera and a rear-facing camera), a Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations:

[0210] The first covariance matrix of the pre-measured echo is determined based on the excitation signal played by the loudspeaker in a quiet state;

[0211] Acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted within the speaker;

[0212] Multiple first audio signals are adaptively filtered based on a reference signal to obtain multiple second audio signals after adaptive filtering.

[0213] Based on the inverse of the first covariance matrix, beamforming is performed on multiple second audio signals to obtain a beamformed third audio signal.

[0214] The third audio signal is subjected to noise cancellation processing to obtain the target audio signal after noise cancellation.

[0215] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0216] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any embodiment of the audio signal processing method provided by the present invention.

[0217] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0218] Since the instructions stored in the storage medium can execute the steps of any embodiment of the audio signal processing method provided in the embodiments of the present invention, the beneficial effects that any audio signal processing method provided in the embodiments of the present invention can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0219] The foregoing has provided a detailed description of an audio signal processing method, apparatus, device, and storage medium according to embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application. Moreover, those skilled in the art can make several improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A method for processing audio signals, applied to an audio interactive device, the audio interactive device comprising multiple microphones and speakers, characterized in that, The method for processing the audio signal includes: Based on the excitation signal played by the loudspeaker in a quiet state, determine the first covariance matrix of the pre-measured echo; Acquire multiple first audio signals containing echoes collected by the multiple microphones, and a reference signal transmitted within the speaker; Based on the reference signal, multiple first audio signals are adaptively filtered to obtain multiple second audio signals after adaptive filtering. Based on the inverse of the first covariance matrix, beamforming is performed on multiple second audio signals to obtain a beamformed third audio signal. The third audio signal is subjected to noise cancellation processing to obtain the target audio signal after noise cancellation; When the azimuth angle of the target audio signal propagation cannot be determined, the step of performing beamforming processing on multiple second audio signals based on the inverse of the first covariance matrix includes: The second weighting coefficients of the minimum variance distortionless response algorithm are determined based on the inverse matrix of the first covariance matrix, the second covariance matrices of the multiple second audio signals, and the second weighting coefficient formula. The second minimum variance distortionless response algorithm is determined based on the second weight coefficient, and beamforming is performed on multiple second audio signals using the second minimum variance distortionless response algorithm. The second formula for calculating the weighting coefficients is: ; in, This represents the covariance matrix of the second audio signal. express The identity matrix, Represents the trace of a matrix. And there are A vector of n elements Let represent the inverse of the first covariance matrix.

2. The audio signal processing method as described in claim 1, characterized in that, The step of adaptively filtering multiple first audio signals according to the reference signal to obtain multiple adaptively filtered second audio signals includes: The echo estimate of the target microphone channel is determined based on the reference signal, and the echo estimate is used as the reference signal for other microphone channels to determine the echo estimate of other microphone channels. Based on the echo estimate of each microphone channel and the first audio signal of the corresponding microphone channel, multiple echo-cancelled second audio signals are obtained; The target microphone can be any one of the plurality of microphones.

3. The audio signal processing method as described in claim 2, characterized in that, The step of obtaining multiple echo-cancelled second audio signals based on the echo estimate of each microphone channel and the first audio signal of the corresponding microphone channel includes: Subtract the echo estimate of the corresponding microphone channel from the first audio signal of each microphone channel to obtain multiple echo-cancelled second audio signals.

4. The audio signal processing method as described in claim 1, characterized in that, The step of determining the first covariance matrix of the pre-measured echo based on the excitation signal played by the loudspeaker in a quiet state includes: Obtain the operating time of the audio interaction device; When the working time meets the preset time condition, the ambient sound is collected through the microphone, and it is detected whether the decibel value of the ambient sound is lower than the preset decibel value. When the ambient sound level is lower than a preset decibel level, the speaker is controlled to play an excitation signal; the excitation signal is a sound signal covering the entire frequency band. Calculate and determine the first covariance matrix of the sound signal.

5. The audio signal processing method as described in claim 4, characterized in that, When the azimuth angle of the target audio signal propagation is determined to be θ, the step of performing beamforming processing on multiple second audio signals based on the inverse of the first covariance matrix includes: The first weighting coefficients of the minimum variance distortionless response algorithm are determined based on the azimuth angle θ, the inverse of the first covariance matrix, and the first weighting coefficient formula. The first minimum variance distortionless response algorithm is determined based on the first weight coefficient, and beamforming processing is performed on multiple second audio signals using the first minimum variance distortionless response algorithm. The first formula for calculating the weighting coefficients is: ; in, Let A be the inverse of the first covariance matrix, and let A be the steering vector determined based on the azimuth angle θ. , Indicates the number of frequency points. , Represents the natural constant Exponentiation, Represents the imaginary unit. , This indicates that the target audio signal has propagated to the first... The time difference between the time of the target audio signal reaching the origin of the coordinate system and the time of the microphone. , It indicates the speed of sound.

6. The audio signal processing method as described in claim 1, characterized in that, The step of performing noise removal processing on the third audio signal to obtain the noise-removed target audio signal includes: The noise in the third audio signal is calculated based on the third audio signal and the noise calculation formula. Based on the noise in the third audio signal, Wiener filtering is performed on the third audio signal to obtain the target audio signal after noise removal; The noise calculation formula is as follows: , As a leakage factor, For frame index, This refers to the third audio signal.

7. An audio signal processing apparatus, applied to an audio interactive device, the audio interactive device comprising multiple microphones and speakers, characterized in that, The audio signal processing device includes: an echo determination module, a signal acquisition module, a first processing module, a second processing module, and an echo cancellation module; The echo determination module is used to determine a first covariance matrix of the pre-measured echo based on the excitation signal played by the speaker in a quiet state. The signal acquisition module is used to acquire multiple first audio signals containing echoes collected by multiple microphones, and a reference signal transmitted in the speaker; The first processing module is configured to perform adaptive filtering on a plurality of first audio signals according to the reference signal to obtain a plurality of adaptively filtered second audio signals; The second processing module is used to perform beamforming processing on multiple second audio signals based on the inverse matrix of the first covariance matrix to obtain a beamformed third audio signal. The echo cancellation module is used to perform noise cancellation processing on the third audio signal to obtain the target audio signal after noise cancellation. The second processing module includes: a second determining unit and a second processing unit; The second determining unit is used to determine the second weighting coefficients of the minimum variance distortionless response algorithm based on the inverse matrix of the first covariance matrix, the second covariance matrix of the multiple second audio signals, and the second weighting coefficient formula when the azimuth angle of the target audio signal propagation cannot be determined. The second processing unit is used to determine a second minimum variance distortionless response algorithm based on the second weight coefficients, and to perform beamforming processing on multiple second audio signals using the second minimum variance distortionless response algorithm; The second formula for calculating the weighting coefficients is: ; in, This represents the covariance matrix of the second audio signal. express The identity matrix, Represents the trace of a matrix. And there are A vector of n elements Let represent the inverse of the first covariance matrix.

8. An audio interactive device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the memory being coupled to the processor, and the processor executing the computer program to implement the method for processing audio signals as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the audio signal processing method as described in any one of claims 1 to 6.

Images