In-vehicle voice processing method, terminal device and storage medium

Abstract

The invention discloses an in-vehicle voice processing method, terminal equipment and a storage medium, and the processing method comprises the steps: obtaining a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone, the distance between the first microphone and the second microphone is smaller than a preset distance; respectively carrying out differential beam processing on the first audio signal and the second audio signal to obtain a first processing signal and a second processing signal; and performing Kalman filtering processing according to the energy of the first processing signal, the second processing signal and the third audio signal at each frequency point, and filtering out audio signals generated in sound areas corresponding to non-microphone in different processing signals or audio signals. According to the scheme, the audio signals which do not belong to the own sound area are filtered out, and the accuracy of in-vehicle voice interaction is improved.

Description

Technical Field

[0001] The present application relates to the field of voice processing technology, and in particular to a method for processing in-vehicle voice, a terminal device, and a storage medium. Background Art

[0002] With the rapid development of science and technology, voice interaction technology is gradually applied to in-vehicle interconnection scenarios. Users are becoming more and more accustomed to interacting with in-vehicle devices through voice, and the requirements and demands for in-vehicle voice interaction systems are increasing. In order to meet the voice interaction between each user in the car and the in-vehicle devices, the in-vehicle voice interaction system has launched the in-vehicle multi-zone voice interaction service to expand the scope of voice interaction.

[0003] Due to the dense crowds and complex environment in the car, the audio signal collected by the onboard microphone corresponding to a certain sound zone not only contains the voice in the current sound zone, but also may include the voice in other sound zones and the audio played by the vehicle itself, etc. If the audio signal is directly used as a voice command, it is easy to cause response errors, which reduces the user experience. Summary of the invention

[0004] The present application provides a method for processing in-vehicle voice, a terminal device and a storage medium.

[0005] A technical solution adopted by the present application is to provide a method for processing in-vehicle voice, wherein at least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone, and the method comprises:

[0006] Acquire a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone; wherein a target distance between the first microphone and the second microphone is less than a preset threshold, and a distance between the third microphone and the first microphone and the second microphone is greater than a preset threshold, respectively;

[0007] Performing differential beam processing on the first audio signal to obtain a first processed signal and performing differential beam processing on the second audio signal to obtain a second processed signal;

[0008] Kalman filtering is performed based on the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point to filter out the audio signal generated by the sound area not corresponding to the first microphone in the first processed signal, filter out the audio signal generated by the sound area not corresponding to the second microphone in the second processed signal, and filter out the audio signal generated by the sound area not corresponding to the third microphone in the third audio signal.

[0009] Optionally, performing Kalman filtering processing according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point includes:

[0010] Performing Fourier transform on the first processed signal to obtain a first frequency domain signal; performing Fourier transform on the second processed signal to obtain a second frequency domain signal; and performing Fourier transform on the third audio signal to obtain a third frequency domain signal;

[0011] The method comprises: performing Kalman filtering processing according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point, filtering out the audio signal generated by the sound area not corresponding to the first microphone in the first processed signal, filtering out the audio signal generated by the sound area not corresponding to the second microphone in the second processed signal, and filtering out the audio signal generated by the sound area not corresponding to the third microphone in the third audio signal, including:

[0012] According to the energy size of each frequency point, Kalman filtering is performed on the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal to filter out the audio signal generated by the sound area not corresponding to the first microphone in the first frequency domain signal, filter out the audio signal generated by the sound area not corresponding to the second microphone in the second frequency domain signal, and filter out the audio signal generated by the sound area not corresponding to the third microphone in the third frequency domain signal to obtain filtered signals corresponding to the sound areas where different microphones are located.

[0013] Optionally, according to the energy of each frequency point, Kalman filtering is performed on the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal to obtain filtered signals corresponding to the sound ranges where different microphones are located, including:

[0014] Determine the target signal from the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal in sequence, and use the remaining signals as reference signals;

[0015] Comparing a first energy of the target signal at each frequency point with a second energy of each reference signal;

[0016] In response to the first energy being less than the second energy, performing Kalman filtering on the target signal using the reference signal to obtain a separation signal corresponding to the target signal at each frequency point;

[0017] In response to the first energy being greater than or equal to the second energy, taking the target signal as a separation signal;

[0018] According to the separated signals at different frequency points, the filtered signals corresponding to the sound ranges where different microphones are located are obtained.

[0019] Optionally, obtaining filtered signals corresponding to different microphones according to separated signals at different frequency points includes:

[0020] Calculate the first frequency domain coherence value between the target signal and the separated signal at each frequency point;

[0021] Calculating a second frequency domain coherence value between the target signal and each reference signal at each frequency point;

[0022] Determine a suppression factor at each frequency point based on a first frequency domain coherence value and a second frequency domain coherence value corresponding to the same frequency point;

[0023] By using the suppression factor and the separated signal, the filtered signals corresponding to different microphones are obtained.

[0024] Optionally, the suppression factor is obtained according to the following formula:

[0025] α=max(0.05,min(1-cohxd,cohed)), where α is the suppression factor, cohed is the first frequency domain coherence value, and cohxd is the second frequency domain coherence value.

[0026] Optionally, the first microphone corresponds to the first sound range, the second microphone corresponds to the second sound range, and the third microphone corresponds to the third sound range;

[0027] After the step of obtaining the audio signal generated by each sound zone according to the filtering signals corresponding to the different sound zones, the method includes:

[0028] Performing an inverse Fourier transform on a first filtered signal corresponding to the first sound range to obtain a first target audio signal generated by the first sound range;

[0029] Performing an inverse Fourier transform on the second filtered signal corresponding to the second sound range to obtain a second target audio signal generated by the second sound range;

[0030] An inverse Fourier transform is performed on the third filtered signal corresponding to the third sound range to obtain a third target audio signal generated by the third sound range.

[0031] Optionally, the first microphone corresponds to the first sound range, and the second microphone corresponds to the second sound range;

[0032] Performing differential beam processing on the first audio signal to obtain a first processed signal, and performing differential beam processing on the second audio signal to obtain a second processed signal, including:

[0033] Acquire a first suppression angle of the first microphone relative to the second sound range, and a second suppression angle of the second microphone relative to the first sound range;

[0034] Using a first suppression angle, differential beam processing is performed on the first audio signal to obtain a first processed signal; and using a second suppression angle, differential beam processing is performed on the second audio signal to obtain a second processed signal.

[0035] Optionally, performing differential beam processing on the first audio signal using the first suppression angle to obtain a first processed signal includes:

[0036] Determining a first filter coefficient according to a first suppression angle, a target distance, and a signal angular frequency;

[0037] Using the first filter coefficient, performing differential beam processing on the first audio signal to obtain a first processed signal;

[0038] Using the second suppression angle, performing differential beam processing on the second audio signal to obtain a second processed signal includes:

[0039] Determining a second filter coefficient according to a second suppression angle, a target distance, and a signal angular frequency;

[0040] The second audio signal is subjected to differential beam processing using the second filter coefficient to obtain a second processed signal.

[0041] Another technical solution adopted by the present application is to provide a terminal device, the terminal device comprising a memory and a processor connected to the memory;

[0042] The memory is used to store program data, and the processor is used to execute the program data to implement the in-vehicle voice processing method as described above.

[0043] Another technical solution adopted by the present application is to provide a computer storage medium, which is used to store program data. When the program data is executed by the computer, it is used to implement the in-vehicle voice processing method as described above.

[0044] The beneficial effects of the present application are: when the distance between the first microphone and the second microphone is less than a preset threshold, a first processed signal and a second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 It is a flowchart of an embodiment of a method for processing in-vehicle speech provided by the present application;

[0047] Figure 2A-2D is a schematic diagram of the distribution positions of vehicle-mounted microphones under different embodiments provided by the present application;

[0048] Figure 3 It is a flowchart of another embodiment of the method for processing in-vehicle speech provided by the present application;

[0049] Figure 4 It is a structural diagram of an embodiment of a terminal device provided by the present application;

[0050] Figure 5 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] In order to improve the riding experience of passengers, smart vehicles are equipped with onboard microphones at different locations inside the vehicle, and personalized services are provided to different passengers by acquiring voice control commands from passengers at different locations. However, due to the dense crowds and complex environment inside the vehicle, the voice signals collected by the onboard microphones include not only the target passenger’s voice, but also interference signals from passengers at other locations, audio signals played by the vehicle itself, and various environmental noises. How to accurately perceive the voice signals of passengers at different locations is an urgent problem to be solved.

[0053] Related technologies mainly use beamforming, blind source separation and other methods. The beamforming algorithm is easily affected by the number and position of the microphone array. For example, it cannot be applied to the layout of linear array and distributed hybrid microphones. The results of blind source separation have the problem of uncertain arrangement. In the speech separation solution based on neural network, there is a certain gap between the simulated room impulse response and the real environment in the car, resulting in distortion of the separated speech.

[0054] This application mainly designs a set of methods to improve the accuracy of separated audio signals. Unlike traditional methods, this application starts from the positional relationship between the vehicle-mounted microphones and combines Kalman filtering to filter out the audio signals that do not belong to the audio zone itself from the signals collected by the vehicle-mounted microphones and mixed with audio generated by different sound zones, thereby accurately identifying users speaking in different sound zones.

[0055] Please refer to Figure 1 , Figure 1It is a flow chart of an embodiment of a method for processing in-vehicle speech provided in the present application.

[0056] like Figure 1 As shown, the method for processing in-vehicle speech in the embodiment of the present application may specifically include the following steps:

[0057] S1, obtaining a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0058] Among them, at least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone.

[0059] The target distance between the first microphone and the second microphone is smaller than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are larger than the preset thresholds respectively.

[0060] like Figure 2A As shown, mic-1 (microphone 1), mic-2 (microphone 2), and mic-3 correspond to the first sound zone, the second sound zone, and the third sound zone, respectively, wherein mic-1 and mic-2 form a linear array microphone array.

[0061] like Figure 2B As shown, mic-1, mic-2, mic-3, and mic-4 correspond to the first sound zone, the second sound zone, the third sound zone, and the fourth sound zone, respectively, wherein mic-1 and mic-2 form a linear array microphone array.

[0062] like Figure 2C As shown, mic-1, mic-2, mic-3, mic-4, mic-5, and mic-6 correspond to the first sound zone, the second sound zone, the third sound zone, the fourth sound zone, the fifth sound zone, and the sixth sound zone, respectively, wherein mic-1 and mic-2 constitute a linear array microphone array.

[0063] like Figure 2D As shown, mic-1, mic-2, mic-3, mic-4, mic-5, and mic-6 correspond to the first sound range, the second sound range, the third sound range, the fourth sound range, the fifth sound range, and the sixth sound range, respectively. Among them, mic-1 and mic-2 constitute a linear array microphone array, and mic-5 and mic-6 constitute a linear array microphone array.

[0064] The method for processing in-vehicle speech provided in the present application is mainly performed by a speech processing device. In some embodiments, the speech processing device may be the vehicle-mounted microphone itself. In some embodiments, the distance measuring device may be a device that is communicatively connected to the vehicle-mounted microphone. For example, the device may be a device for monitoring images, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and an autonomous vehicle, a robot, a security system, or any one or more products of glasses and helmets for augmented reality or virtual reality. In some possible implementations, the method for processing in-vehicle speech may be implemented by a processor calling computer-readable instructions stored in a memory.

[0065] Specifically, the speech processing device obtains a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0066] Exemplarily, the first microphone corresponds to the first sound zone, and the second microphone corresponds to the second sound zone. The distance between the first microphone and the second microphone is less than a preset threshold, that is, the first microphone and the second microphone constitute a linear array microphone array (or simply a microphone array). Since the distance between the first microphone and the second microphone is relatively close, the first audio signal collected by the first microphone includes not only the audio generated in the first sound zone, but also the audio generated in the second sound zone. Similarly, the second audio signal collected by the second microphone includes the audio generated in the first sound zone and the second sound zone. After a period of research, the applicant has found that directly performing Kalman filtering operations on the first audio signal and the second audio signal cannot obviously separate the audio signal that does not belong to the own sound zone. Based on this, it is necessary to perform relevant processing on the first audio signal and the second audio signal to amplify the difference between the first audio signal and the second audio signal.

[0067] In some embodiments, the preset threshold is less than 50 cm. In some embodiments, the preset threshold is less than 40 cm. In some embodiments, the preset threshold is less than 30 cm. In some embodiments, the preset threshold is less than 20 cm. In some embodiments, the preset threshold is less than 10 cm. Exemplarily, the preset threshold is 15 cm.

[0068] S2, performing differential beam processing on the first audio signal to obtain a first processed signal, and performing differential beam processing on the second audio signal to obtain a second processed signal.

[0069] Specifically, the speech processing device performs differential beam processing on the first audio signal to obtain a first processed signal, and the speech processing device performs differential beam processing on the second audio signal to obtain a second processed signal.

[0070] In the related technology, beamforming is mainly used. However, the application scenario corresponding to this technology is that the first microphone and the second microphone constitute an additive microphone array, and the additive microphone array determines the shape of its main lobe by setting the main lobe itself. For the first microphone and the second microphone belonging to the vehicle-mounted microphone array, the distance between the microphones is small, and the width of the main lobe of the beam is large, resulting in poor directivity, and the array gain is not obvious. The enhancement effect on the target direction is weak, and the suppression effect on the interference direction is poor.

[0071] The differential beamforming algorithm is a sound source localization and sound separation technology based on microphone arrays, which reflects the difference in spatial sound pressure. Compared with related technologies, the differential microphone array determines the shape of its main lobe by setting the null point direction, and by placing the null point in the interference direction, the speech energy ratio in the target direction and the interference direction is increased. The sensitivity of the array to different incident directions is represented by the beam pattern, and the differential beam satisfies the following relationship:

[0072] B[h(ω),θ]=d H (ω,cosθ)h(ω)

[0073] Where d(ω, cosθ) is the steering vector of the array, ω is the signal angular frequency, θ is the incident direction, h(ω) is the output weight of each microphone, and H is the conjugate transpose. Taking the dual microphone end-fire direction as 0° and the suppression direction as 180° as an example, the beam pattern calculation formula for differential beamforming is:

[0074]

[0075] Where h′(ω) is τ is the ratio of the microphone array spacing to the speed of sound.

[0076] In this embodiment, the speech processing device applies the differential beam algorithm to the audio enhancement of the real vehicle linear array microphone array. For example, the speech processing device sets the zero point to the angle of the second sound zone relative to the microphone array, thereby effectively suppressing the audio energy in the direction of the second sound zone, thereby highlighting the audio signal of the first sound zone, and enhancing the energy difference of the audio signals generated by the first sound zone and the second sound zone.

[0077] For another example, the speech processing device sets the zero point to the angle of the first sound zone relative to the microphone array, thereby effectively suppressing the audio energy in the direction of the first sound zone, thereby highlighting the audio signal of the second sound zone.

[0078] In this embodiment, the first audio signal and the second audio signal are processed by the differential beam algorithm, and the energy difference between the obtained first processed signal and the second processed signal becomes larger, and the signal energy distribution is similar to the sound pickup result of a distributed microphone. A distributed microphone refers to two microphones whose spacing is greater than or equal to a preset threshold.

[0079] Exemplarily, the first microphone corresponds to the first sound zone, and the second microphone corresponds to the second sound zone. Since the first microphone and the second microphone are close to each other, the energy difference of the speech signals in the first sound zone and the second sound zone picked up by the speech processing device is small, which affects the effect of subsequent Kalman filtering and causes more speech damage. Therefore, differential beam processing is first performed on the first audio signal and the second audio signal corresponding to the compactly arranged first microphone and the second microphone, respectively, to reduce the sound of the second sound zone received by the first microphone and reduce the sound of the first sound zone received by the second microphone through directional interference suppression.

[0080] S3, performing Kalman filtering according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point, filtering out the audio signal generated by the sound area not corresponding to the first microphone in the first processed signal, filtering out the audio signal generated by the sound area not corresponding to the second microphone in the second processed signal, and filtering out the audio signal generated by the sound area not corresponding to the third microphone in the third audio signal.

[0081] Assuming that each frequency point corresponds to only one speaker, it is understandable that the signal energy collected by the microphone closest to the sound area is the largest. Therefore, it is judged whether the current sound area is speaking based on the energy size at the frequency point.

[0082] Exemplarily, the speech processing device performs Kalman filtering on the first processed signal, the second processed signal, and the third processed signal according to the energy between the first processed signal, the second processed signal, and the third audio signal at each frequency point, filters out the audio signal generated by the sound zone that does not correspond to the first microphone in the first processed signal, filters out the audio signal generated by the sound zone that does not correspond to the second microphone in the second processed signal, and filters out the audio signal generated by the sound zone that does not correspond to the third microphone in the third audio signal.

[0083] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0084] Another embodiment of the in-vehicle speech processing method provided by the present application may specifically include the following steps:

[0085] S11, acquiring a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0086] Among them, at least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone.

[0087] The target distance between the first microphone and the second microphone is smaller than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are larger than the preset thresholds respectively.

[0088] S12, performing differential beam processing on the first audio signal to obtain a first processed signal, and performing differential beam processing on the second audio signal to obtain a second processed signal.

[0089] S13, performing Fourier transform on the first processed signal to obtain a first frequency domain signal; performing Fourier transform on the second processed signal to obtain a second frequency domain signal; and performing Fourier transform on the third audio signal to obtain a third frequency domain signal.

[0090] Since the first processed signal, the second processed signal and the third audio signal are time domain signals, in the subsequent steps, Kalman filtering needs to be performed according to the energy of the signals in different frequency domains, and therefore, a conversion operation from the time domain signal to the frequency domain signal is required.

[0091] Exemplarily, the speech processing apparatus may perform short-time Fourier transform (STFT) on the first processed signal, the second processed signal and the third audio signal, respectively, to obtain a first frequency domain signal, a second frequency domain signal and a third frequency domain signal.

[0092] S14, performing Kalman filtering processing on the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal according to the energy size of each frequency point, filtering out the audio signal generated by the sound zone that does not correspond to the first microphone in the first frequency domain signal, filtering out the audio signal generated by the sound zone that does not correspond to the second microphone in the second frequency domain signal, filtering out the audio signal generated by the sound zone that does not correspond to the third microphone in the third frequency domain signal, and obtaining filtered signals corresponding to the sound zones where different microphones are located.

[0093] Specifically, the speech processing device performs Kalman filtering on the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal according to the energy size of each frequency point, filters out the audio signal generated by the sound area not corresponding to the first microphone in the first frequency domain signal, filters out the audio signal generated by the sound area not corresponding to the second microphone in the second frequency domain signal, and filters out the audio signal generated by the sound area not corresponding to the third microphone in the third frequency domain signal, to obtain filtered signals corresponding to the sound areas where different microphones are located.

[0094] Exemplarily, the first microphone corresponds to a first filtered signal, the second microphone corresponds to a second filtered signal, and the third microphone corresponds to a third filtered signal.

[0095] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0096] Another embodiment of the in-vehicle speech processing method provided by the present application may specifically include the following steps:

[0097] S21, acquiring a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0098] Among them, at least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone.

[0099] The target distance between the first microphone and the second microphone is smaller than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are larger than the preset thresholds respectively.

[0100] S22, performing differential beam processing on the first audio signal to obtain a first processed signal, and performing differential beam processing on the second audio signal to obtain a second processed signal.

[0101] S23, performing Fourier transform on the first processed signal to obtain a first frequency domain signal; performing Fourier transform on the second processed signal to obtain a second frequency domain signal; and performing Fourier transform on the third audio signal to obtain a third frequency domain signal.

[0102] S24, determining a target signal from the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal in sequence, and taking the remaining signals as reference signals.

[0103] In some embodiments, the speech processing device uses the first frequency domain signal as the target signal, and uses the second frequency domain signal and the third frequency domain signal as reference signals in sequence.

[0104] In some embodiments, the speech processing device uses the second frequency domain signal as a target signal, and uses the first frequency domain signal and the fourth frequency domain signal as reference signals in sequence.

[0105] In some embodiments, the speech processing device uses the third frequency domain signal as the target signal, and uses the first frequency domain signal and the second frequency domain signal as reference signals in sequence.

[0106] S25, comparing the first energy of the target signal at each frequency point with the second energy of each reference signal.

[0107] Specifically, the speech processing device traverses all frequency points, obtains the energy corresponding to the target signal at each frequency point (ie, the first energy), and the energy corresponding to the reference signal (ie, the second energy). Further, the speech processing device compares the magnitude relationship between the first energy and the second energy.

[0108] S26, in response to the first energy being less than the second energy, performing Kalman filtering on the target signal using the reference signal to obtain a separation signal corresponding to the target signal at each frequency point.

[0109] Specifically, at the first target frequency point, in response to the first energy being less than the second energy, it indicates that speech is generated in the sound area corresponding to the reference signal. The speech processing device performs Kalman filtering on the target signal using the reference signal to obtain a separated signal corresponding to the target signal at the first target frequency point.

[0110] Among them, the separation signal satisfies the following relationship:

[0111]

[0112] Among them, d n is the target signal, x n is the reference signal, w nis the Kalman filter coefficient, se n The separated signal output after filtering can also be called an error signal.

[0113] In some embodiments, the separated signal may be a single frequency point or a partial frequency band interval.

[0114] S27, in response to the first energy being greater than or equal to the second energy, taking the target signal as a separation signal.

[0115] Specifically, in response to the first energy being greater than or equal to the second energy, it indicates that speech may have been generated in the sound area corresponding to the target signal, and the speech processing device does not perform any operation.

[0116] S28, obtaining filtered signals corresponding to the sound ranges where different microphones are located according to the separated signals at different frequency points.

[0117] In some embodiments, the first frequency domain signal is a target signal, the second frequency domain signal is a reference signal, and the speech processing device determines the first candidate filter signal based on the first separated signal (point) at different frequency points. Further, the speech processing device uses the first candidate filter signal as the target signal and the third frequency domain signal as the reference signal, executes steps S25-S27, obtains the second separated signal (point) at different frequency points, and thereby determines the filter signal corresponding to the sound zone where the first microphone is located, that is, the filter signal corresponding to the first sound zone.

[0118] Similarly, the filtered signal corresponding to the second sound range and the filtered signal corresponding to the third sound range can also be obtained in the above manner.

[0119] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0120] Another embodiment of the in-vehicle speech processing method provided by the present application may specifically include the following steps:

[0121] S31, acquiring a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0122] Among them, at least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone.

[0123] The target distance between the first microphone and the second microphone is smaller than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are larger than the preset thresholds respectively.

[0124] S32, performing differential beam processing on the first audio signal to obtain a first processed signal, and performing differential beam processing on the second audio signal to obtain a second processed signal.

[0125] S33, performing Fourier transform on the first processed signal to obtain a first frequency domain signal; performing Fourier transform on the second processed signal to obtain a second frequency domain signal; and performing Fourier transform on the third audio signal to obtain a third frequency domain signal.

[0126] S34, determining a target signal from the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal in sequence, and taking the remaining signals as reference signals.

[0127] S35, comparing the first energy of the target signal at each frequency point with the second energy of each reference signal.

[0128] S36, in response to the first energy being less than the second energy, performing Kalman filtering on the target signal using the reference signal to obtain a separation signal corresponding to the target signal at each frequency point.

[0129] S37, in response to the first energy being greater than or equal to the second energy, taking the target signal as a separation signal.

[0130] S38, calculating a first frequency domain coherence value between the target signal and the separated signal at each frequency point.

[0131] Specifically, the speech processing device obtains the target separation signal corresponding to the target microphone, and then calculates the first frequency domain coherence value between the target signal and the separation signal at each frequency point.

[0132] The first frequency domain coherence value is used to describe the similarity between the target signal and the separated signal at the current frequency point. The higher the first frequency domain coherence value is, the closer the energy values ​​of the target signal and the separated signal are at the frequency point.

[0133] In some embodiments, the first frequency domain coherence value satisfies the following relationship:

[0134]

[0135] Where D(f) is the target signal and E(f) is the separation signal.

[0136] S39, calculating a second frequency domain coherence value between the target signal and each reference signal at each frequency point.

[0137] The second frequency domain coherence value is used to describe the similarity between the target signal and a reference signal at the current frequency point. The higher the second frequency domain coherence value is, the closer the energy value of the target signal and the reference signal is at the frequency point.

[0138] In some embodiments, the second frequency domain coherence value satisfies the following relationship:

[0139]

[0140] Where D(f) is the target signal and X(f) is the reference signal.

[0141] S40, determining a suppression factor at each frequency point based on the first frequency domain coherence value and the second frequency domain coherence value corresponding to the same frequency point.

[0142] Specifically, the speech processing device calculates the suppression factor corresponding to each frequency point by using the first frequency domain coherence value and the second frequency domain coherence value at each frequency point.

[0143] In some embodiments, the inhibition factor is obtained according to the following formula:

[0144] α=max(0.05,min(1-cohxd,cohed))

[0145] Among them, α is the suppression factor, cohed is the first frequency domain coherence value, and cohxd is the second frequency domain coherence value.

[0146] S41, using the suppression factor and the separation signal, obtaining filter signals corresponding to different microphones.

[0147] Specifically, the speech processing device calculates the product of the suppression factor at each frequency point and the target separation signal to obtain the target filter signal corresponding to the target microphone, and so on to obtain the filter signals corresponding to different microphones.

[0148] It should be noted that the Kalman filter can only eliminate the content in the target signal that is relevant to the reference signal, and there will be some residues in the separated signal. Therefore, it is necessary to further remove the residual interference signal through the above steps.

[0149] In some embodiments, S38 - S41 may also be referred to as non-linear processing.

[0150] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0151] In addition, considering the limitations of the Kalman filter, the present application further performs post-processing on the result after the Kalman filter processing to further remove the residual interference signal in the speech signal.

[0152] Another embodiment of the in-vehicle speech processing method provided by the present application may specifically include the following steps:

[0153] S51, acquiring a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0154] Among them, at least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone.

[0155] The target distance between the first microphone and the second microphone is smaller than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are larger than the preset thresholds respectively.

[0156] S52, performing differential beam processing on the first audio signal to obtain a first processed signal, and performing differential beam processing on the second audio signal to obtain a second processed signal.

[0157] S53, performing Fourier transform on the first processed signal to obtain a first frequency domain signal; performing Fourier transform on the second processed signal to obtain a second frequency domain signal; and performing Fourier transform on the third audio signal to obtain a third frequency domain signal.

[0158] S54, determine the target signal from the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal in sequence, and use the remaining signals as reference signals.

[0159] S55, comparing the first energy of the target signal at each frequency point with the second energy of each reference signal.

[0160] S56, in response to the first energy being less than the second energy, performing Kalman filtering on the target signal using the reference signal to obtain a separation signal corresponding to the target signal at each frequency point.

[0161] S57, in response to the first energy being greater than or equal to the second energy, taking the target signal as a separation signal.

[0162] S58, obtaining the filtered signals corresponding to the sound ranges where different microphones are located according to the separated signals at different frequency points.

[0163] S59, performing an inverse Fourier transform on the first filtered signal corresponding to the first sound zone to obtain a first target audio signal generated by the first sound zone.

[0164] In some embodiments, the first filtered signal is a time domain signal. It is understandable that the time domain signal is difficult to directly apply to subsequent voice interaction tasks such as voice wake-up and voice recognition. Therefore, it is necessary to convert the first filtered signal belonging to the time domain signal into the first target audio signal belonging to the frequency domain signal.

[0165] Exemplarily, the speech processing device performs short-time Fourier transform on the first processed signal, and the speech processing device performs short-time inverse Fourier transform on the first filtered signal.

[0166] S60, performing an inverse Fourier transform on the second filtered signal corresponding to the second sound range to obtain a second target audio signal generated by the second sound range.

[0167] In some embodiments, the second filtered signal is a time domain signal. It is understandable that the time domain signal is difficult to directly apply to subsequent voice interaction tasks such as voice wake-up and voice recognition. Therefore, it is necessary to convert the second filtered signal belonging to the time domain signal into a second target audio signal belonging to the frequency domain signal.

[0168] Exemplarily, the speech processing device performs short-time Fourier transform on the second processed signal, and the speech processing device performs short-time inverse Fourier transform on the second filtered signal.

[0169] S61, performing an inverse Fourier transform on a third filtered signal corresponding to the third sound zone to obtain a third target audio signal generated by the third sound zone.

[0170] In some embodiments, the third filtered signal is a time domain signal. It is understandable that the time domain signal is difficult to directly apply to subsequent voice interaction tasks such as voice wake-up and voice recognition. Therefore, it is necessary to convert the third filtered signal belonging to the time domain signal into a third target audio signal belonging to the frequency domain signal.

[0171] Exemplarily, the speech processing device performs short-time Fourier transform on the third processed signal, and the speech processing device performs short-time inverse Fourier transform on the third filtered signal.

[0172] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0173] Another embodiment of the in-vehicle speech processing method provided by the present application may specifically include the following steps:

[0174] S71, acquiring a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0175] At least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone. The first microphone corresponds to the first sound zone, and the second microphone corresponds to the second sound zone.

[0176] The target distance between the first microphone and the second microphone is smaller than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are larger than the preset thresholds respectively.

[0177] S72, obtaining a first suppression angle of the first microphone relative to the second sound range, and a second suppression angle of the second microphone relative to the first sound range.

[0178] Specifically, in response to the target distance between the first microphone and the second microphone being less than a preset threshold, the speech processing device determines that the first microphone and the second microphone constitute a linear array microphone. Further, the speech processing device determines a first suppression angle θ of the first microphone relative to the second sound zone from a large amount of pre-acquired sound pickup data of the first microphone and the second microphone collected from real vehicles and theoretical modeling results. 2 , and a second suppression angle θ of the second microphone relative to the first sound zone 1 .

[0179] S73, performing differential beam processing on the first audio signal using the first suppression angle to obtain a first processed signal; and performing differential beam processing on the second audio signal using the second suppression angle to obtain a second processed signal.

[0180] Specifically, the speech processing device uses the first suppression angle θ 2, performing differential beam processing on the first audio signal to obtain a first processed signal.

[0181] And, the speech processing device uses the second suppression angle θ 1 , perform differential beam processing on the second audio signal to obtain a second processed signal.

[0182] S74, performing Kalman filtering processing according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point, filtering out the audio signal generated by the sound area not corresponding to the first microphone in the first processed signal, filtering out the audio signal generated by the sound area not corresponding to the second microphone in the second processed signal, and filtering out the audio signal generated by the sound area not corresponding to the third microphone in the third audio signal.

[0183] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0184] In addition, the in-vehicle speech processing method provided in the present application utilizes a differential beam processing method to increase the energy difference of the sound zone signals collected by compactly arranged microphones, thereby improving the filtering effect of subsequent Kalman filtering.

[0185] Another embodiment of the in-vehicle speech processing method provided by the present application may specifically include the following steps:

[0186] S81, acquiring a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, and a third audio signal collected by a third microphone.

[0187] At least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, and each microphone corresponds to a sound zone. The first microphone corresponds to the first sound zone, and the second microphone corresponds to the second sound zone.

[0188] The target distance between the first microphone and the second microphone is smaller than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are larger than the preset thresholds respectively.

[0189] S82, obtaining a first suppression angle of the first microphone relative to the second sound range, and a second suppression angle of the second microphone relative to the first sound range.

[0190] S83, determining a first filter coefficient according to the first suppression angle, the target distance, and the signal angular frequency.

[0191] In some embodiments, the speech processing device calculates the ratio of the target distance to the speed of sound to obtain a target ratio.

[0192] Furthermore, the speech processing device determines a first filter coefficient according to the first suppression angle, the target ratio, and the signal angular frequency.

[0193] Among them, the first filter coefficient satisfies the following relationship:

[0194]

[0195] Wherein, C1 is the first filter coefficient.

[0196] S84: Perform differential beam processing on the first audio signal using the first filter coefficient to obtain a first processed signal.

[0197] Specifically, the speech processing device performs differential beam processing on the first audio signal using the first filter coefficient to obtain a first processed signal.

[0198] S85, determining a second filter coefficient according to the second suppression angle, the target distance, and the signal angular frequency.

[0199] In some embodiments, the speech processing device determines the second filter coefficient according to the second suppression angle, the target coefficient, and the signal angular frequency.

[0200] Among them, the second filter coefficient satisfies the following relationship:

[0201]

[0202] Wherein, C2 is the first filter coefficient.

[0203] S86: Perform differential beam processing on the second audio signal using the second filter coefficient to obtain a second processed signal.

[0204] Specifically, the speech processing device performs differential beam processing on the second audio signal using the second filter coefficient to obtain a second processed signal.

[0205] S87, performing Kalman filtering processing according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point, filtering out the audio signal generated by the sound zone that does not correspond to the first microphone in the first processed signal, filtering out the audio signal generated by the sound zone that does not correspond to the second microphone in the second processed signal, and filtering out the audio signal generated by the sound zone that does not correspond to the third microphone in the third audio signal.

[0206] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0207] In addition, the in-vehicle speech processing method provided in the present application utilizes a differential beam processing method to increase the energy difference of the sound zone signals collected by compactly arranged microphones, thereby improving the filtering effect of subsequent Kalman filtering.

[0208] See also Figure 3 , Figure 3 It is a flowchart of another embodiment of the in-vehicle voice processing method provided by the present application.

[0209] like Figure 3 As shown, another embodiment of the in-vehicle speech processing method provided by the present application may specifically include the following steps:

[0210] S101, acquiring a first audio signal collected by a first microphone, a second audio signal collected by a second microphone, a third audio signal collected by a third microphone, and a fourth audio signal collected by a fourth microphone.

[0211] The first microphone and the second microphone form a linear microphone array, the third microphone does not form a linear microphone array with other microphones, and the fourth microphone does not form a linear microphone array with other microphones.

[0212] In this embodiment, the layout of the vehicle microphone is as follows: Figure 2B shown.

[0213] S102: Perform differential beam processing on the first audio signal and the second audio signal respectively to obtain a first processed signal corresponding to the first audio signal and a second processed signal corresponding to the second audio signal.

[0214] Since the first microphone and the second microphone are close to each other, the energy difference between the speech signals corresponding to the first sound range and the second sound range picked up by the two microphones is small, which affects the effect of subsequent Kalman filtering and causes more speech damage.

[0215] Therefore, the speech processing device processes the first audio signals s respectively corresponding to the first microphone and the second microphone which are arranged closely. 1 and the second audio signal s 2 , perform corresponding differential beam processing, and weaken the sound of the second sound zone received by the first microphone and the sound of the first sound zone received by the second microphone by directional interference suppression, and obtain the corresponding first processed signal s′ 1 and the second processed signal s′ 2 .

[0216] Through the above steps, the energy difference of the signals in different sound zones collected by the vehicle-mounted microphone is increased.

[0217] S103, performing Fourier transform on the first processed signal to obtain a first frequency domain signal; performing Fourier transform on the second processed signal to obtain a second frequency domain signal; performing Fourier transform on the third audio signal to obtain a third frequency domain signal; performing Fourier transform on the fourth audio signal to obtain a fourth frequency domain signal.

[0218] Specifically, the speech processing device processes the first processed signal s′ 1 , the second processed signal s′ 2 , the third audio signal s 3 , the fourth audio signal s 4 Perform Fourier transform to obtain the first frequency domain signal X 1 , the second frequency domain signal X 2 , the third frequency domain signal X 3 and the fourth frequency domain signal X 4 , realizing the transformation of time domain signal to frequency domain signal.

[0219] S104, at each frequency point, perform Kalman filtering on the first frequency domain signal, the second frequency domain signal, the third frequency domain signal, and the fourth frequency domain signal, respectively, to obtain corresponding first filtered signals, second filtered signals, third filtered signals, and fourth filtered signals.

[0220] Specifically, the Kalman filter performed in this embodiment satisfies the following relationship:

[0221]

[0222] Among them, d n is the target signal, x n is the reference signal, w nis the Kalman filter coefficient, se n is the error signal, that is, the filtered signal output after filtering.

[0223] Assuming that there is only one speaker at each frequency point, the signal energy collected by the microphone closest to the sound area is the largest. Therefore, the energy at the frequency point is used to determine whether the current sound area is speaking, thereby determining whether to perform filtering operations on the frequency point. Each microphone signal is filtered once with all other microphone signals as the separation result of the sound area.

[0224] In some application scenarios, S104 may specifically include the following steps:

[0225] The speech processing device converts the first frequency domain signal X 1 As the target signal, the second frequency domain signal X 2 , the third frequency domain signal X 3 , the fourth frequency domain signal X 4 They are used as reference signals respectively and traverse all frequency points.

[0226] If the energy of the reference signal is greater than the energy of the target signal at a certain frequency point, it can be considered that the speech is in the voice zone corresponding to the reference signal, and the speech processing device performs a Kalman filter operation on the target signal. Otherwise, it is considered that the speech is in the voice zone corresponding to the target signal, and the speech processing device does not perform any operation, thereby obtaining the separation signal se of the first voice zone. 1 .

[0227] Similarly, the speech processing device converts the second frequency domain signal X 2 As the target signal, the first frequency domain signal X 1 , the third frequency domain signal X 3 , the fourth frequency domain signal X 4 As reference signals, repeat the above steps to obtain the separation signal se of the second sound zone. 2 .

[0228] Similarly, the speech processing device converts the third frequency domain signal X 3 As the target signal, the first frequency domain signal X 1 , the second frequency domain signal X 2 , the fourth frequency domain signal X 4 As reference signals, repeat the above steps to obtain the separation signal se of the third sound zone. 3 .

[0229] Similarly, the speech processing device converts the fourth frequency domain signal X 4 As the target signal, the first frequency domain signal X 1 , the second frequency domain signal X 2 , the third frequency domain signal X 3As reference signals, repeat the above steps to obtain the separation signal se of the passenger and rear seats. 4 .

[0230] S105, at each frequency point, nonlinearly process the first filter signal, the second filter signal, the third filter signal and the fourth filter signal respectively to obtain a speech separation result corresponding to each sound zone.

[0231] The Kalman filter can only eliminate the content in the target signal that is relevant to the reference signal, and some residues will remain. To overcome the above problem, the speech processing device calculates the target signal d at each frequency point. n With the error signal se n , target signal d n With the reference signal x n The frequency domain coherence value of is calculated as follows:

[0232]

[0233] Among them, D(f), X(f), and E(f) correspond to the target signal, reference signal, and error signal respectively. The nonlinearity is reflected in the calculation of the suppression factor based on the coherence value, and the calculation formula is as follows:

[0234] α=max(0.05,min(1-cohxd,cohed))

[0235] Furthermore, the speech processing device calculates the product of the suppression factor and the error signal, determines the speech separation result S1 corresponding to the first sound area, the speech separation result S2 corresponding to the second sound area, the speech separation result S3 corresponding to the third sound area, and the speech separation result S4 corresponding to the fourth sound area, so as to suppress the residual signal.

[0236] Among them, the above-mentioned speech separation results are all time domain signals.

[0237] In the above scheme, the distance between the first microphone and the second microphone is less than a preset threshold value, and the first processed signal and the second processed signal are obtained by performing differential beam processing on the first audio signal collected by the first microphone and the second audio signal collected by the second microphone, so as to increase the difference between the audio signals collected by the microphones with smaller spacing. At each frequency point, the audio signal is Kalman filtered according to the energy difference between the audio signals, and the signals generated by other sound zones in the audio signal are filtered out, so as to accurately separate the voice signal. Furthermore, the in-car voice processing method provided by the present application can further improve the accuracy of voice interaction of in-car users and enhance the user experience.

[0238] In addition, the in-vehicle speech processing method provided in the present application utilizes a differential beam processing method to increase the energy difference of the sound zone signals collected by compactly arranged microphones, thereby improving the filtering effect of subsequent Kalman filtering.

[0239] Please continue to see Figure 4 , Figure 4 The terminal device 500 of the embodiment of the present application includes a processor 51 and a memory 52 .

[0240] The processor 51 and the memory 52 are connected to the bus. The memory 52 stores program data. The processor 51 is used to execute the program data to implement the in-vehicle speech processing method described in the above embodiment.

[0241] In the embodiment of the present application, the processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having the ability to process signals. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 51 may also be any conventional processor, etc.

[0242] This application also provides a computer storage medium, please continue to refer to Figure 5 , Figure 5 It is a structural diagram of an embodiment of a computer storage medium provided in the present application. The computer storage medium 600 stores program data 61. When the program data 61 is executed by the processor, it is used to implement the in-vehicle voice processing method of the above embodiment.

[0243] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0244] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Equivalent structures or equivalent process changes made by utilizing the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for processing in-vehicle speech, characterized in that: At least a first microphone, a second microphone and a third microphone are arranged inside the vehicle, each microphone corresponds to a sound zone, and the method includes: Acquire a first audio signal collected by the first microphone, a second audio signal collected by the second microphone, and a third audio signal collected by the third microphone; wherein the target distance between the first microphone and the second microphone is less than a preset threshold, and the distances between the third microphone and the first microphone and the second microphone are respectively greater than the preset threshold; Performing differential beam processing on the first audio signal to obtain a first processed signal and performing differential beam processing on the second audio signal to obtain a second processed signal; Kalman filtering is performed according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point to filter out audio signals generated by the sound area not corresponding to the first microphone in the first processed signal, filter out audio signals generated by the sound area not corresponding to the second microphone in the second processed signal, and filter out audio signals generated by the sound area not corresponding to the third microphone in the third audio signal.

2. The method according to claim 1, characterized in that The performing Kalman filtering processing according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point includes: Performing Fourier transform on the first processed signal to obtain a first frequency domain signal; performing Fourier transform on the second processed signal to obtain a second frequency domain signal; and performing Fourier transform on the third audio signal to obtain a third frequency domain signal; The method of performing Kalman filtering according to the energy of the first processed signal, the second processed signal, and the third audio signal at each frequency point, filtering out the audio signal generated by the sound zone not corresponding to the first microphone in the first processed signal, filtering out the audio signal generated by the sound zone not corresponding to the second microphone in the second processed signal, and filtering out the audio signal generated by the sound zone not corresponding to the third microphone in the third audio signal, comprises: According to the energy size of each frequency point, Kalman filtering is performed on the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal to filter out audio signals generated by the sound zone not corresponding to the first microphone in the first frequency domain signal, filter out audio signals generated by the sound zone not corresponding to the second microphone in the second frequency domain signal, and filter out audio signals generated by the sound zone not corresponding to the third microphone in the third frequency domain signal to obtain filtered signals corresponding to the sound zones where different microphones are located.

3. The method according to claim 2, characterized in that The step of performing Kalman filtering on the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal according to the energy of each frequency point to obtain filter signals corresponding to the sound ranges where different microphones are located includes: Determine a target signal from the first frequency domain signal, the second frequency domain signal, and the third frequency domain signal in sequence, and use the remaining signals as reference signals; Comparing a first energy of the target signal at each frequency point with a second energy of each reference signal; In response to the first energy being less than the second energy, performing Kalman filtering on the target signal using the reference signal to obtain a separated signal corresponding to the target signal at each frequency point; In response to the first energy being greater than or equal to the second energy, using the target signal as the separation signal; The filtered signals corresponding to the sound ranges where different microphones are located are obtained according to the separated signals at different frequency points.

4. The method according to claim 3, characterized in that The step of obtaining the filtered signals corresponding to different microphones according to the separated signals at different frequency points includes: Calculating a first frequency domain coherence value between the target signal and the separated signal at each frequency point; Calculating a second frequency domain coherence value between the target signal and each reference signal at each frequency point; Determine a suppression factor at each frequency point based on the first frequency domain coherence value and the second frequency domain coherence value corresponding to the same frequency point; The filtering signals corresponding to different microphones are obtained by using the suppression factor and the separated signal.

5. The method according to claim 4, characterized in that The inhibition factor is obtained according to the following formula: α=max(0.05,min(1-cohxd,cohed)), wherein α is the suppression factor, cohed is the first frequency domain coherence value, and cohxd is the second frequency domain coherence value.

6. The method according to claim 3, characterized in that The first microphone corresponds to the first sound range, the second microphone corresponds to the second sound range, and the third microphone corresponds to the third sound range; After the step of obtaining the filtered signals corresponding to the sound ranges where different microphones are located according to the separated signals at different frequency points, the method further comprises: Performing an inverse Fourier transform on a first filtered signal corresponding to the first sound zone to obtain a first target audio signal generated by the first sound zone; Performing an inverse Fourier transform on a second filtered signal corresponding to the second sound range to obtain a second target audio signal generated by the second sound range; An inverse Fourier transform is performed on the third filtered signal corresponding to the third sound zone to obtain a third target audio signal generated by the third sound zone.

7. The method according to claim 1, characterized in that The first microphone corresponds to the first sound range, and the second microphone corresponds to the second sound range; The performing differential beam processing on the first audio signal to obtain a first processed signal, and performing differential beam processing on the second audio signal to obtain a second processed signal, comprises: Acquire a first suppression angle of the first microphone relative to the second sound range, and a second suppression angle of the second microphone relative to the first sound range; The first audio signal is differentially processed by using the first suppression angle to obtain the first processed signal; and the second audio signal is differentially processed by using the second suppression angle to obtain the second processed signal.

8. The method according to claim 7, characterized in that The step of performing differential beam processing on the first audio signal by using the first suppression angle to obtain the first processed signal includes: Determining a first filter coefficient according to the first suppression angle, the target distance, and the signal angular frequency; Using the first filter coefficient, performing differential beam processing on the first audio signal to obtain the first processed signal; The step of performing differential beam processing on the second audio signal by using the second suppression angle to obtain the second processed signal includes: determining a second filter coefficient according to the second suppression angle, the target distance, and the signal angular frequency; The second audio signal is subjected to differential beam processing using the second filter coefficient to obtain the second processed signal.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory connected to the processor, wherein: The memory stores program instructions; The processor is configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The storage medium stores program instructions, and when the program instructions are executed, the method according to any one of claims 1 to 8 is implemented.

Images