Vehicle-mounted sound area separation method, device, equipment and medium

Abstract

The invention discloses a vehicle-mounted sound area separation method and device, equipment and a medium. And determining a multi-channel noise reduction mask through a pre-trained noise reduction network. The multi-channel noise reduction mask can accurately indicate an effective signal segment and a noise signal segment in the echo-removed vehicle-mounted array signal. Based on the multi-channel noise reduction mask, a pure audio signal, namely a multi-channel noise reduction array signal, can be further extracted, so that the interference of noise on a subsequent sound area separation task is reduced, and a sound area separation model can identify and process the audio signal more accurately. Through the noise reduction network, the sound area separation task and the noise reduction task of the sound area separation model can be decoupled, and the situation that the sound area separation model directly processes an audio signal with too high noise, so that the model is over-fitted to noise features, and the accuracy of sound area separation is reduced is avoided.

Description

Technical Field

[0001] The present application relates to the field of digital signal processing and deep learning technology, and in particular to a vehicle-mounted sound zone separation method, device, equipment and medium. Background Art

[0002] With the continuous advancement of smart cockpit technology, audio zone separation technology, as one of its core technologies, has increasingly become the key to improving the in-car audio experience and interactive performance. Audio zone separation technology mainly processes the in-car array data to achieve accurate separation of the sound source of each seat, thereby supporting the in-car zone audio control technology. This technology not only significantly improves the clarity of the in-car audio, but also provides a solid foundation for realizing more intelligent in-car interaction.

[0003] In the existing sound zone separation system architecture, the original multi-channel audio signal will first be processed by the echo cancellation module to eliminate the interference of the sound played by the speakers in the car on the original audio signal. Subsequently, the audio signal after echo cancellation will be sent to the sound zone separation module, and the sound sources of different sound zones will be separated through the sound zone separation algorithm (or called the sound zone separation model). The sound zone separation algorithm is usually trained by simulating the audio data and noise data of each sound zone in the car and setting the corresponding multi-sound zone separation output data.

[0004] However, although this mainstream technical solution can perform well under low noise levels, when the noise inside the car is strong, the noise will cause strong interference to the phase information that the sound zone separation task relies on, resulting in problems such as sound zone separation errors or poor noise reduction effects. Summary of the invention

[0005] The present application provides a vehicle-mounted sound zone separation method, device, equipment and medium, which are used to solve the problem that the existing sound zone separation method cannot effectively separate the noise inside the vehicle, resulting in sound zone separation errors or poor noise reduction effect.

[0006] In a first aspect, the present application provides a method for separating vehicle-mounted audio zones, the method comprising:

[0007] Obtain multi-channel echo-free vehicle array signals;

[0008] Determine a multi-channel noise reduction mask based on the de-echoed vehicle array signal through a pre-trained noise reduction network; wherein the multi-channel noise reduction mask is used to indicate a valid signal segment and a noise signal segment in the de-echoed vehicle array signal;

[0009] Based on the multi-channel noise reduction mask and the echo-removed vehicle-mounted array signal, obtaining a multi-channel noise reduction array signal;

[0010] The pre-trained sound zone separation model is used to determine the audio signal after the sound zone separation based on the multi-channel noise reduction array signal.

[0011] In a second aspect, the present application further provides a vehicle-mounted audio zone separation device, the device comprising:

[0012] An acquisition unit, used for acquiring multi-channel echo-free vehicle array signals;

[0013] A denoising unit, configured to determine a multi-channel denoising mask based on the de-echoed vehicle array signal through a pre-trained denoising network; wherein the multi-channel denoising mask is used to indicate a valid signal segment and a noise signal segment in the de-echoed vehicle array signal; and to obtain a multi-channel denoising array signal based on the multi-channel denoising mask and the de-echoed vehicle array signal;

[0014] The sound zone separation unit is used to determine the audio signal after sound zone separation based on the multi-channel noise reduction array signal through a pre-trained sound zone separation model.

[0015] In a third aspect, the present application provides a computer device, the computer device comprising a processor, the processor being configured to implement the steps of the vehicle-mounted sound zone separation method as described above when executing a computer program stored in a memory.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the vehicle-mounted sound zone separation method as described above.

[0017] The beneficial effects of this application are as follows:

[0018] 1. Determine a multi-channel noise reduction mask through a pre-trained noise reduction network. The multi-channel noise reduction mask can accurately indicate the valid signal segment and the noise signal segment in the de-echoed vehicle array signal. Based on the multi-channel noise reduction mask, a pure audio signal, i.e., a multi-channel noise reduction array signal, can be further extracted, thereby reducing the interference of noise on the subsequent sound zone separation task, so that the sound zone separation model can more accurately identify and process the audio signal.

[0019] 2. Since the multi-channel noise reduction array signal has removed most of the noise, the subsequent sound zone separation model can more accurately identify the audio content of different sound zones when processing the multi-channel noise reduction array signal. This helps to improve the accuracy of sound zone separation, so that the audio signals of different sound zones can be clearly separated.

[0020] 3. Through the denoising network, the voice zone separation task and the denoising task of the voice zone separation model can be decoupled to avoid the voice zone separation model directly processing audio signals with too much noise, causing the model to overfit to the noise characteristics and reduce the accuracy of voice zone separation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 A schematic diagram of a vehicle audio zone separation process provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a training process of a specific voice zone separation model provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of a specific process of separating vehicle audio zones provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of a specific noise reduction process provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of a vehicle-mounted audio zone separation device provided in an embodiment of the present application;

[0027] Figure 6 It is a structural schematic diagram of a computer device provided in an optional embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0029] In order to improve the quality and noise reduction effect of sound zone separation results in scenarios with strong in-vehicle noise, the present application provides a vehicle-mounted sound zone separation method, device, equipment and medium.

[0030] Embodiment 1:

[0031] The present application provides a method for separating vehicle-mounted audio zones. Figure 1 A schematic diagram of a vehicle audio zone separation process provided in an embodiment of the present application, the process includes:

[0032] S101: Acquire multi-channel echo-free vehicle array signals.

[0033] The vehicle-mounted audio zone separation method provided in the present application is applied to a computer device, which may be an intelligent terminal, such as an in-vehicle central control, an in-vehicle processor, etc., or a server, such as an in-vehicle business server, etc.

[0034] In the present application, sensors (such as microphones, speakers, etc.) for collecting audio signals have been arranged in array form and installed in the target vehicle in advance according to the size and shape of the target vehicle to ensure that sound signals from all directions inside the target vehicle can be accurately captured. For example, microphones are installed in sound zones such as around the driver's seat, in the passenger area, and even in the trunk to achieve comprehensive monitoring of all important sound sources in the vehicle. Later, during the operation of the target vehicle, audio signals from various areas in the vehicle can be continuously collected through microphones in different areas inside the target vehicle (recorded as multi-channel array signals in the vehicle). The computer device for performing vehicle-mounted sound zone separation can process the multi-channel array signal in the vehicle based on the collected multi-channel array signal in the vehicle, thereby removing the noise contained in the multi-channel array signal in the vehicle and achieving sound zone separation of the multi-channel array signal in the vehicle.

[0035] In the in-vehicle communication system, line echo is a common problem that reduces the clarity and intelligibility of calls. Line echo is usually caused by the sound emitted by the speaker inside the vehicle being reflected inside the vehicle and then captured by the microphone again. By performing line echo removal processing on the collected in-vehicle multi-channel array signal, the quality of sound zone separation can be significantly improved. Based on this, in the present application, after the in-vehicle multi-channel array signal is obtained based on the above embodiment, the in-vehicle multi-channel array signal can be subjected to line echo removal processing to obtain a multi-channel de-echoed in-vehicle array signal. For example, the in-vehicle multi-channel array signal is pre-processed using adaptive filtering technology, or the in-vehicle multi-channel array signal is processed using AES and RES technology.

[0036] In a possible implementation manner, the step of acquiring a multi-channel echo-free vehicle array signal includes:

[0037] Receive multi-channel array signals and speaker signals in the car;

[0038] According to the in-vehicle speaker signal, the in-vehicle multi-channel array signal is subjected to echo removal processing to obtain the multi-channel echo-free vehicle array signal.

[0039] The in-car speaker signal is an audio signal played through the vehicle audio system, which is usually used for in-car entertainment, navigation prompts, etc. It is one of the sources of echo generation and has an important impact on the echo component in the in-car multi-channel array signal. Based on this, in the present application, the in-car speaker signal can also be obtained. For example, by connecting to the interface of the vehicle audio system of the target vehicle, it is ensured that the audio content played by the in-car speaker can be obtained in real time. The in-car speaker signal is used as a reference signal to perform echo removal on the in-car multi-channel array signal. Specifically, after obtaining the in-car speaker signal and the in-car multi-channel array signal, the correlation between the in-car speaker signal and the in-car multi-channel array signal can be analyzed. This correlation can be expressed as a feature based on the audio signal in time and space. For example, when the speaker plays a sound, each microphone in the microphone array will capture the sound signal, but the time and intensity of these signals reaching each microphone may be different. By analyzing these differences, the propagation path and reflection of the audio signal inside the vehicle can be inferred. Then, the filter parameters are dynamically adjusted according to this correlation, thereby adapting to the changes in the audio signal in real time to filter out or suppress unnecessary frequency components of the multi-channel array signal in the car, that is, suppressing the echo components in the multi-channel array signal in the car, thereby improving the quality of the audio signal.

[0040] In a possible implementation, the speech features of the multi-channel echo-free vehicle array signal can be determined, and the speech features can be subsequently processed to achieve processing of the echo-free vehicle array signal, so as to reduce the amount of calculation required for subsequent processing. The speech features may include Mel frequency cepstrum coefficients, spectrograms, Mel filter energy, short-time Fourier transform coefficients, pitch, timbre, rhythm, etc. For example, the multi-channel echo-free vehicle array signal is framed, windowed, and short-time Fourier transformed in parallel, so as to convert the multi-channel echo-free vehicle array signal into multi-channel short-time Fourier transform complex features.

[0041] S102: Determine a multi-channel noise reduction mask based on the de-echoed vehicle array signal through a pre-trained noise reduction network; wherein the multi-channel noise reduction mask is used to indicate a valid signal segment and a noise signal segment in the de-echoed vehicle array signal.

[0042] After obtaining the de-echoed vehicle array signal based on the above embodiment, the de-echoed vehicle array signal can be processed by a pre-trained noise reduction network to determine a multi-channel noise reduction mask. The multi-channel noise reduction mask is used to indicate the valid signal segment and the noise signal segment in the de-echoed vehicle array signal. Exemplarily, the multi-channel de-echoed vehicle array signal can be directly input into the noise reduction network. The noise reduction network performs noise reduction processing on the signal of each channel respectively and outputs the corresponding multi-channel noise reduction mask.

[0043] In a possible implementation, determining a multi-channel noise reduction mask based on the de-echoed vehicle array signal using a pre-trained noise reduction network includes:

[0044] Performing mean processing on the echo-removed vehicle-mounted array signal to obtain a single-channel mean array signal;

[0045] Determining a single-channel denoising mask based on the single-channel mean array signal by the denoising network;

[0046] Based on the single-channel noise reduction mask, the multi-channel noise reduction mask is determined.

[0047] In order to improve processing efficiency and accuracy, and reduce the amount of calculation of the denoising network. The multi-channel echo-removed vehicle array signal can be averaged to obtain a single-channel average array signal. This single-channel average array signal can be regarded as the average or synthesis of the audio signals of multiple channels. It contains the common information in all channels while reducing the differences and redundancy between channels. Then, the obtained single-channel average array signal is input into the denoising network. The denoising network performs denoising based on this single-channel signal and outputs a denoising mask. Although the denoising mask is single-channel, it can be applied to all original multi-channel audio signals because it reflects the common noise pattern in all channels. Based on this, a multi-channel denoising mask can be generated according to the single-channel denoising mask output by the denoising network. For example, the denoising mask for each channel can be obtained by copying the single-channel denoising mask to each channel or fine-tuning the single-channel denoising mask according to the correlation between channels.

[0048] For example, the multi-channel echo-free vehicle array signal is X i (t, f), where i represents the channel, t represents the time, and f represents the frequency band. The echo-free vehicle array signals of all channels are averaged to obtain a single-channel mean signal X(t, f). The single-channel mean signal X(t, f) is sent to the denoising network for processing. The denoising network performs denoising on the single-channel mean signal X(t, f) and extracts a single-channel denoising mask M(t, f) from the single-channel mean signal X(t, f).

[0049] S103: Acquire a multi-channel noise reduction array signal based on the multi-channel noise reduction mask and the echo-removed vehicle-mounted array signal.

[0050] Each element in the multi-channel noise reduction mask obtained based on the above embodiment corresponds to a channel and time point, and its value (usually between 0 and 1) represents the probability or weight that the audio signal at that position is a valid signal. The multi-channel noise reduction mask is then applied to the echo-removed vehicle array signal to obtain a multi-channel noise reduction array signal. In this way, signal segments that are considered to be noise can be weakened (because the mask value is close to 0), while valid signal segments can be retained or enhanced (because the mask value is close to 1).

[0051] Illustratively, for each channel and time point, the signal value is multiplied by the corresponding mask value.

[0052] For example, after the single-channel noise reduction mask M(t, f) is obtained based on the above embodiment, the single-channel noise reduction mask M(t, f) is applied to the echo-free vehicle array signal X of each channel. i (t, f), a multi-channel noise reduction array signal is obtained. It is specifically expressed by the following formula:

[0053]

[0054] in, represents the denoised signal value of the echo-free vehicle array signal of the i-th channel at the t-th time point and the f-th frequency band, M(t,f) represents the single-channel denoising mask, X i (t,f) represents the signal value of the echo-free vehicle array signal of the i-th channel at the t-th time point and the f-th frequency band.

[0055] In a possible implementation, when collecting multi-channel array signals in a vehicle, there may be phase differences between audio signals of different channels due to differences in physical positions between microphones, differences in sound propagation paths, and environmental factors. This phase inconsistency will cause errors in subsequent signal processing (such as sound zone separation), affecting the quality of the final audio signal. Based on this, in the present application, after acquiring the multi-channel noise reduction array signal, the multi-channel noise reduction array signal may be corrected for phase consistency between channels. For example, the phase difference between different channels is calculated, and then the phase of the signal of one or more channels is adjusted according to the calculated phase difference, so that the signals of all channels remain consistent in phase. The phase difference between different channels can be eliminated through phase consistency correction, so that the sound zone separation model can more accurately identify and separate audio signals of different sound zones, reduce signal distortion and interference caused by phase inconsistency, and thus improve the quality of the final audio signal.

[0056] S104: Determine an audio signal after the sound zone separation based on the multi-channel noise reduction array signal by using a pre-trained sound zone separation model.

[0057] In the present application, a sound zone separation model is pre-trained, and the sound zone separation model can identify and separate audio signals of different sound zones in the audio signal. After the multi-channel noise reduction array signal is obtained based on the above embodiment, the multi-channel noise reduction array signal can be input into the pre-trained sound zone separation model. The sound zone separation model can process the input multi-channel noise reduction array signal and output audio signals of different sound zones. Among them, the sound zone may include one or more of the following: main driver sound zone, co-driver sound zone, main driver rear sound zone and co-driver rear sound zone.

[0058] The beneficial effects of this application are as follows:

[0059] 1. Determine a multi-channel noise reduction mask through a pre-trained noise reduction network. The multi-channel noise reduction mask can accurately indicate the valid signal segment and the noise signal segment in the de-echoed vehicle array signal. Based on the multi-channel noise reduction mask, a pure audio signal, i.e., a multi-channel noise reduction array signal, can be further extracted, thereby reducing the interference of noise on the subsequent sound zone separation task, so that the sound zone separation model can more accurately identify and process the audio signal.

[0060] 2. Since the multi-channel noise reduction array signal has removed most of the noise, the subsequent sound zone separation model can more accurately identify the audio content of different sound zones when processing the multi-channel noise reduction array signal. This helps to improve the accuracy of sound zone separation, so that the audio signals of different sound zones can be clearly separated.

[0061] 3. Through the denoising network, the voice zone separation task and the denoising task of the voice zone separation model can be decoupled to avoid the voice zone separation model directly processing audio signals with too much noise, causing the model to overfit to the noise characteristics and reduce the accuracy of voice zone separation.

[0062] Embodiment 2:

[0063] In order to accurately reduce noise, based on the above embodiments, in the present application, the noise reduction network is obtained by training the original noise reduction network through multi-channel noise-free frequency samples in the car in the noise-free frequency sample set and multi-channel noise audio samples in the car in the noise audio sample set.

[0064] In the present application, a noise-free frequency sample set and a noise audio sample set can be collected in advance to train the original noise reduction network through these two sample sets to obtain a trained noise reduction network. Among them, the noise-free frequency sample set contains a variety of in-car multi-channel noise-free frequency samples. These in-car multi-channel noise-free frequency samples are pure and do not contain any background noise or interference. They represent the clear and accurate audio content that is expected to be heard in an actual vehicle environment, such as human voices, music, etc. The noise audio sample set contains a variety of in-car multi-channel noise audio samples, and these in-car multi-channel noise audio samples cover various noise types that may be encountered in an actual vehicle environment, such as background noise, wind noise, road noise, engine noise, etc.

[0065] It should be noted that the noise audio sample set should include as many types of in-car multi-channel noise audio samples as possible to improve the robustness of the trained noise reduction network.

[0066] In the process of training the original noise reduction network, any in-car multi-channel noise-free audio sample can be obtained from the noise-free audio sample set, and at least one in-car multi-channel noise audio sample can be obtained from the noise audio sample set. Then the in-car multi-channel audio sample is mixed with the in-car multi-channel noise audio sample to obtain the in-car multi-channel noisy audio sample. The in-car multi-channel noisy audio sample is input into the original noise reduction network. Through the original noise reduction network, the input in-car multi-channel noisy audio sample is processed to obtain the expected in-car multi-channel noise-reduced audio. According to the expected in-car multi-channel noise-reduced audio and the in-car multi-channel noise-free audio sample, a loss value is determined. According to the loss value, the parameters in the original noise reduction network are adjusted to obtain a trained noise reduction network.

[0067] The original denoising network may be an untrained network or a general denoising model that has been trained with a large number of noisy audio samples and their corresponding clean audio samples, and the general denoising model has a general denoising capability.

[0068] Through the above embodiment, a number of in-vehicle multi-channel noisy frequency samples can be obtained. For each in-vehicle multi-channel noisy frequency sample, the above operation is performed to train the original noise reduction network. When the preset convergence condition is met, the original noise reduction network training is completed, and the trained noise reduction network of the target model vehicle is obtained.

[0069] Among them, the preset convergence condition can be that the sum of the loss values ​​determined based on each in-vehicle multi-channel noisy frequency sample is less than a pre-configured loss threshold, or the sum of the determined loss values ​​has been in a downward trend and tends to be flat, or the number of iterations for training the original denoising network reaches the set maximum number of iterations, etc. In specific implementations, it can be flexibly set and is not specifically limited here.

[0070] As a possible implementation method, when training the original noise reduction network, the multi-channel noisy frequency samples in the car can be divided into training samples and test samples. The original noise reduction network is first trained based on the training samples, and then the reliability of the trained noise reduction network is verified based on the test samples.

[0071] Embodiment 3:

[0072] In order to train the sound zone separation model supporting the target vehicle, based on the above embodiments, in the present application, a clean voice sample library and a noisy in-vehicle impulse response database of the vehicle model (recorded as the target vehicle model) are collected in advance, so that the original model is iteratively trained through the clean voice sample library and the noisy in-vehicle impulse response database to obtain the trained sound zone separation model supporting the sound zone separation task of the target vehicle. Among them, the noisy in-vehicle impulse response database includes a large number of multi-channel noise samples related to the target vehicle, and the multi-channel noise samples include simulated multi-channel noise data and collected multi-channel noise data, and the clean voice sample library includes a large number of multi-channel audio samples, and the multi-channel audio samples include simulated multi-channel audio data and a batch of pre-collected collected multi-channel audio data.

[0073] In one example, for each type of multi-channel audio sample, a method for collecting each type of multi-channel audio sample is described respectively:

[0074] Type 1, analog multi-channel audio data.

[0075] In this application, the acoustic simulation algorithm can be used to create simulated multi-channel audio data in various vehicle operation scenarios, such as voice signals of different speakers in different areas in vehicle operation scenarios such as engine start, acceleration, deceleration, braking, door closing, etc. These simulated data can be provided in a multi-channel format to ensure that stereo effects and spatial sound information are included, so that the audio recognition system can better understand and process the sounds inside and around the vehicle.

[0076] In some possible implementations, the acoustic characteristics of a vehicle may change as the number of people in the vehicle changes. For example, more passengers in the vehicle may reduce the noise level in the vehicle because the human body can absorb some of the sound. Therefore, the distribution of people that may exist in the target model vehicle can be considered, and the simulated multi-channel audio data of the target model vehicle can be simulated to ensure that the collected simulated multi-channel audio data is closer to the acoustic characteristics of the multi-channel voice signal collected in the target model vehicle in the real world, thereby preventing the network from collapsing. Exemplarily, the probability of the distribution of the number of people in the target model vehicle can be obtained, and then according to the probability of the distribution of the number of people, the simulated multi-channel audio data of the target model vehicle under different numbers of people can be simulated and generated.

[0077] In a possible implementation, detailed simulated multi-channel audio data is generated for each specific configuration of the number of people using acoustic simulation algorithms, such as finite element analysis, boundary element analysis, mirror source, etc. For example, the room impulse response of the space is synthesized using the mirror source method, and the simulated multi-channel audio data of the target model vehicle is obtained by combining the pre-collected near-field clean voice signal.

[0078] It should be noted that for each population distribution, different sound source positions and sound propagation paths need to be considered to ensure the authenticity of the simulated multi-channel audio data.

[0079] Type 2: Collect multi-channel audio data.

[0080] Method A: When collecting multi-channel audio data in the target vehicle, the data can be collected with the In-Car Communication (ICC) system turned on to ensure that the collected real audio data can ensure that the test conditions match the actual use conditions, thereby helping to collect data that truly reflects the performance of the in-car communication system. Among them, the ICC system is a communication technology designed for the interior of the vehicle. It uses micro-amplifiers, speakers and advanced signal processing technology to capture the voice of the front row of the vehicle and clearly transmit it to the rear passengers, while reducing the interference of environmental noise.

[0081] Mode B, can also obtain and collect multi-channel audio data (denoted as the first multi-channel audio data) by mixing the single-speaker audio data corresponding to different areas in the target model vehicle. Exemplary, pre-configure the recording environment of the single-speaker in the target model vehicle, ensure that other noise factors inside the target model vehicle are reduced as much as possible, such as turning off the engine, audio equipment, and isolating external traffic noise. Multiple microphones are set inside the target model vehicle, and are respectively located at different sound areas inside the target model vehicle, such as the driver's seat, the co-pilot seat, the back seat, etc., to collect audio samples from different positions. When each speaker is sampled by audio, the speaker can be allowed to speak at different positions of the target model vehicle, and the audio data (denoted as the single-speaker audio data) of the speaker in each sound area are recorded. Subsequently, when obtaining the first multi-channel audio data, the single-speaker audio data corresponding to different sound areas and from different speakers can be obtained, and then by audio synthesis technology, the multiple single-speaker audio data obtained are mixed, thereby obtaining the first multi-channel audio data.

[0082] It should be noted that when the audio samples were taken for each speaker, the in-car communication system of the target model vehicle was not turned on.

[0083] In one example, for each type of multi-channel noise sample, methods for collecting each type of multi-channel noise sample are described respectively:

[0084] Type A, collect multi-channel noise data.

[0085] In order to improve the anti-interference ability of the sound zone separation model, in the present application, noise signals that may appear can be collected in advance for each working scene of the target model vehicle, and these noise signals can be determined as multi-channel noise samples according to the types of these noise signals, such as white noise, Gaussian noise, salt and pepper noise, etc. Exemplarily, the classification of different scenes inside the target model vehicle is determined, and these scenes may include but are not limited to: stationary state, slow driving, fast driving, highway driving, driving in rainy days, in-car environment with passengers talking, etc. Each scene has its own unique noise characteristics. Multiple microphones are set inside the target model vehicle, respectively located in different sound areas inside the target model vehicle, such as the driver's seat, the co-driver's seat, the back seat, etc., to collect noise signals from different positions.

[0086] Type B, simulated multi-channel noise data.

[0087] In this application, the acoustic simulation algorithm can also be used to create simulated multi-channel noise data of the target model vehicle in different scenarios, such as interference signals in different areas of the vehicle in vehicle operation scenarios such as engine start, acceleration, deceleration, braking, and door closing, so as to ensure that the types of multi-channel noise samples are as many as possible, thereby improving the robustness and accuracy of the subsequent trained sound zone separation model. For example, simulated multi-channel noise data can be generated by the mirror source method.

[0088] Figure 2 A schematic diagram of the training process of a specific sound zone separation model provided in an embodiment of the present application. Through a data simulation method, multi-channel audio samples in a clean speech sample library related to a target vehicle and multi-channel noise samples in a noise in-vehicle impact response database related to the target vehicle are mixed to simulate noisy array speech with data. Then, feature extraction is performed on the noisy array speech to obtain noisy array speech features, and the noisy array speech features are fed into a neural network model to estimate the gain Mask of the spectrum. Next, the estimated gain Mask is multiplied by the spectrum of the noisy array speech to estimate the spectrum of the predicted clean speech. Finally, the loss between the estimated clean speech spectrum and the actual clean speech spectrum is calculated, and the loss is fed back to the neural network model for adjusting the model parameters. This process is iterated continuously until the loss is no longer significantly reduced, thereby achieving optimization of the model.

[0089] Embodiment 4:

[0090] The vehicle-mounted audio zone separation method provided by the present application is described below through specific embodiments. Figure 3 A schematic diagram of a specific process of separating vehicle audio zones provided in an embodiment of the present application, the process includes:

[0091] S301: Receive a multi-channel array signal and a speaker signal in the vehicle.

[0092] S302: According to the in-vehicle speaker signal, the in-vehicle multi-channel array signal is subjected to echo removal processing to obtain a multi-channel echo-free in-vehicle array signal.

[0093] S303: Determine a multi-channel noise reduction mask based on the de-echoed vehicle array signal using a pre-trained noise reduction network.

[0094] The multi-channel noise reduction mask is used to indicate the valid signal segment and the noise signal segment in the echo-removed vehicle array signal.

[0095] In a possible implementation, the noise reduction network is obtained by training the original noise reduction network using multi-channel noise-free audio samples in the vehicle in the noise-free audio sample set and multi-channel noise audio samples in the vehicle in the noise audio sample set.

[0096] S304: Acquire a multi-channel noise reduction array signal based on the multi-channel noise reduction mask and the echo-removed vehicle array signal.

[0097] S305: Perform inter-channel phase consistency correction on the multi-channel noise reduction array signal.

[0098] S306: Determine an audio signal after the sound zone separation based on the multi-channel noise reduction array signal by using a pre-trained sound zone separation model.

[0099] The specific process of S303 to S305 is described below through a specific embodiment. Figure 4 A schematic diagram of a specific noise reduction process provided in an embodiment of the present application, the process includes:

[0100] S401: performing mean processing on the acquired echo-removed vehicle-mounted array signal to obtain a single-channel mean array signal.

[0101] S402: Determine a single-channel denoising mask based on a single-channel mean array signal through a denoising network.

[0102] S403: Determine a multi-channel noise reduction mask based on the single-channel noise reduction mask.

[0103] S404: Acquire a multi-channel noise reduction array signal based on the multi-channel noise reduction mask and the echo-removed vehicle array signal.

[0104] Embodiment 5:

[0105] Based on the same inventive concept, the present application also provides a vehicle-mounted audio zone separation device, Figure 5 A schematic diagram of the structure of a vehicle-mounted audio zone separation device provided in an embodiment of the present application, the device comprising:

[0106] An acquisition unit 51 is used to acquire a multi-channel echo-free vehicle array signal;

[0107] A denoising unit 52 is configured to determine a multi-channel denoising mask based on the de-echoed vehicle array signal through a pre-trained denoising network; wherein the multi-channel denoising mask is used to indicate a valid signal segment and a noise signal segment in the de-echoed vehicle array signal; and obtain a multi-channel denoising array signal based on the multi-channel denoising mask and the de-echoed vehicle array signal;

[0108] The sound zone separation unit 53 is used to determine the audio signal after sound zone separation based on the multi-channel noise reduction array signal by using a pre-trained sound zone separation model.

[0109] In some possible implementations, the noise reduction unit 52 is specifically used to perform mean processing on the de-echoed vehicle-mounted array signal to obtain a single-channel mean array signal; determine a single-channel noise reduction mask based on the single-channel mean array signal through the noise reduction network; and determine the multi-channel noise reduction mask based on the single-channel noise reduction mask.

[0110] In some possible implementations, the acquisition unit 51 is specifically used to receive an in-vehicle multi-channel array signal and an in-vehicle horn signal; and perform echo removal processing on the in-vehicle multi-channel array signal according to the in-vehicle horn signal to obtain the multi-channel echo-free vehicle array signal.

[0111] In some possible implementations, the noise reduction unit 52 is further configured to perform inter-channel phase consistency correction on the multi-channel noise reduction array signal after acquiring the multi-channel noise reduction array signal.

[0112] In some possible implementations, the noise reduction network is obtained by training the original noise reduction network using multi-channel noise-free audio samples in the vehicle in the noise-free audio sample set and multi-channel noise audio samples in the vehicle in the noise audio sample set.

[0113] The vehicle-mounted audio zone separation device in this embodiment is presented in the form of a functional module, where the module refers to an application specific integrated circuit (ASIC), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0114] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0115] Embodiment 6:

[0116] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present application, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0117] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0118] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0119] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0121] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0122] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0123] Embodiment 7:

[0124] On the basis of the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program executable by a processor is stored. When the program runs on the processor, the processor implements the following steps when executing:

[0125] Obtain multi-channel echo-free vehicle array signals;

[0126] Determine a multi-channel noise reduction mask based on the de-echoed vehicle array signal through a pre-trained noise reduction network; wherein the multi-channel noise reduction mask is used to indicate a valid signal segment and a noise signal segment in the de-echoed vehicle array signal;

[0127] Based on the multi-channel noise reduction mask and the echo-removed vehicle-mounted array signal, obtaining a multi-channel noise reduction array signal;

[0128] The pre-trained sound zone separation model is used to determine the audio signal after the sound zone separation based on the multi-channel noise reduction array signal.

[0129] Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to that of the vehicle-mounted audio zone separation method, the implementation of the above-mentioned computer-readable storage medium can refer to the embodiment of the method, and the repeated parts will not be repeated.

[0130] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A vehicle-mounted sound zone separation method, characterized in that: The method comprises: Obtain multi-channel echo-free vehicle array signals; Determine a multi-channel noise reduction mask based on the de-echoed vehicle array signal through a pre-trained noise reduction network; wherein the multi-channel noise reduction mask is used to indicate a valid signal segment and a noise signal segment in the de-echoed vehicle array signal; Based on the multi-channel noise reduction mask and the echo-removed vehicle-mounted array signal, obtaining a multi-channel noise reduction array signal; The pre-trained sound zone separation model is used to determine the audio signal after the sound zone separation based on the multi-channel noise reduction array signal.

2. The method according to claim 1, characterized in that The method of determining a multi-channel noise reduction mask based on the de-echoed vehicle array signal through a pre-trained noise reduction network includes: Performing mean processing on the echo-removed vehicle-mounted array signal to obtain a single-channel mean array signal; Determining a single-channel denoising mask based on the single-channel mean array signal by the denoising network; Based on the single-channel noise reduction mask, the multi-channel noise reduction mask is determined.

3. The method according to claim 1, characterized in that The method of obtaining a multi-channel echo-free vehicle array signal includes: Receive multi-channel array signals and speaker signals in the car; According to the in-vehicle speaker signal, the in-vehicle multi-channel array signal is subjected to echo removal processing to obtain the multi-channel echo-free in-vehicle array signal.

4. The method according to claim 1, characterized in that After acquiring the multi-channel noise reduction array signal, and before determining the audio signal after the sound zone separation based on the multi-channel noise reduction array signal by using the pre-trained sound zone separation model, the method further includes: Inter-channel phase consistency correction is performed on the multi-channel noise reduction array signal.

5. The method according to claim 1, characterized in that The noise reduction network is obtained by training the original noise reduction network through multi-channel noise-free audio samples in the car in the noise-free audio sample set and multi-channel noise audio samples in the car in the noise audio sample set.

6. A vehicle-mounted sound zone separation device, characterized in that: The device comprises: An acquisition unit, used for acquiring multi-channel echo-free vehicle array signals; A denoising unit, configured to determine a multi-channel denoising mask based on the de-echoed vehicle array signal through a pre-trained denoising network; wherein the multi-channel denoising mask is used to indicate a valid signal segment and a noise signal segment in the de-echoed vehicle array signal; and to obtain a multi-channel denoising array signal based on the multi-channel denoising mask and the de-echoed vehicle array signal; The sound zone separation unit is used to determine the audio signal after sound zone separation based on the multi-channel noise reduction array signal through a pre-trained sound zone separation model.

7. The device according to claim 6, characterized in that The noise reduction unit is specifically used to perform mean processing on the echo-removed vehicle array signal to obtain a single-channel mean array signal; and determine a single-channel noise reduction mask based on the single-channel mean array signal through the noise reduction network; Based on the single-channel noise reduction mask, the multi-channel noise reduction mask is determined.

8. The device according to claim 6, characterized in that The noise reduction unit is further used to perform inter-channel phase consistency correction on the multi-channel noise reduction array signal after acquiring the multi-channel noise reduction array signal.

9. A computer device, characterized in that: The computer device includes a processor, and the processor is used to implement the steps of the vehicle-mounted sound zone separation method as described in any one of claims 1 to 5 when executing a computer program stored in a memory.

10. A computer-readable storage medium, characterized in that: It stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle-mounted sound zone separation method as described in any one of claims 1 to 5 are implemented.

Images