Vehicle-mounted multi-zone speech separation method, electronic device, and storage medium

Abstract

The present invention discloses a vehicle-mounted multi-zone speech separation method, an electronic device and a storage medium, wherein a vehicle-mounted multi-zone speech separation method includes: convolving acquired high-fidelity audio with acquired room impulse response data to obtain a mixed signal and at least one speech label; training a fusion beamforming network model based on the mixed signal and the at least one speech label; testing the fusion beamforming network model based on a preset simulation test set to determine whether the fusion beamforming network model meets preset requirements; if the preset requirements are met, predicting the beamforming weights of the mixed signal and the at least one speech label based on the fusion beamforming network model to obtain each sound zone separation signal.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of speech recognition technology, and in particular to a vehicle-mounted multi-zone speech method, electronic device, and storage medium. Background Art

[0002] With the development of intelligent cockpits in cars, the requirements for in-vehicle multi-zone voice interaction are becoming increasingly higher, especially in multi-zone voice separation.

[0003] In the existing technology, the traditional algorithm for in-vehicle multi-zone speech separation mainly involves technologies such as acoustic echo cancellation (AEC), beamforming (BF), blind source separation (BSS), and post-processing (POST). AEC is mainly used as a pre-processing for speech separation to eliminate the local playback and TTS voice broadcast collected by the microphone in the in-vehicle scenario; BF enhances each sound zone through the phase information between the microphones; BSS is based on the independence assumption of the signal source and demixes the signal into several independent components through the statistical distribution; POST mainly suppresses the residual interference in each channel after separation. The disadvantage is that the processing process is cumbersome and it is difficult for each module to achieve the optimal effect at the same time, resulting in poor overall separation effect.

[0004] The NN-based in-vehicle multi-zone speech separation solution also includes the AEC part, but compared to traditional solutions, BF, BSS, and POST are replaced by end-to-end NN (neural network) solutions. End-to-end NN solutions are mainly divided into two categories: time domain-based and frequency domain-based. The frequency domain-based NN solution can be divided into mask-based, such as ideal binary mask (IBM) or ideal ratio mask (IRM), and mapping-based methods, such as logarithmic power spectrum (LPS) or magnitude spectrum (MS), according to different optimization objectives. The defect is that there is a certain gap between the simulated room impulse response (RIR) and the actual RIR in the car, resulting in large distortion of the multi-zone separation audio directly output by the NN, which has a greater impact on the back-end such as recognition and wake-up. Summary of the Invention

[0005] The embodiments of the present invention provide a vehicle-mounted multi-zone speech separation method and device, which are used to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a method for vehicle-mounted multi-zone speech separation, comprising: convolving the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech label; training a fusion beamforming network model based on the mixed signal and the at least one speech label; testing the fusion beamforming network model based on a preset simulation test set to determine whether the fusion beamforming network model meets preset requirements; if the preset requirements are met, predicting the beamforming weights of the mixed signal and the at least one voice label based on the fusion beamforming network model to obtain each sound zone separation signal.

[0007] In a second aspect, an embodiment of the present invention provides an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the above-mentioned vehicle-mounted multi-zone speech separation methods of the present invention.

[0008] In a third aspect, an embodiment of the present invention provides a storage medium, in which one or more programs including execution instructions are stored. The execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute any of the above-mentioned vehicle-mounted multi-zone speech separation methods of the present invention.

[0009] In a fourth aspect, an embodiment of the present invention further provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any one of the above-mentioned vehicle-mounted multi-zone speech separation methods.

[0010] The method of the present application uses three-dimensional modeling software to obtain real vehicle data based on the three-dimensional model of the vehicle, thereby achieving higher accuracy than real vehicle measurement and conducting model training in advance; then a beamforming network module is used to predict the beamforming weight based on a large amount of prior information, thereby greatly reducing the fuzzy area between sound zones, allowing users to adopt a more free sitting posture to interact in the car, and improving the resolution and accuracy of sound zone separation; further, the separation signals of each sound zone are used to guide the separation of traditional blind source separation, thereby achieving a higher degree of separation. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flowchart of a vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention;

[0012] Figure 2A flowchart of another vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention;

[0013] Figure 3 A flowchart of another vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention;

[0014] Figure 4 A flowchart of another vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention;

[0015] Figure 5 A schematic diagram of the structure of a dual-tone zone of a specific example of a vehicle-mounted multi-tone zone speech separation method provided by an embodiment of the present invention;

[0016] Figure 6 A block diagram of a vehicle-mounted multi-zone speech separation design, which is a specific example of a vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention;

[0017] Figure 7 A flowchart of a vehicle-mounted multi-zone speech separation method according to a specific example of a vehicle-mounted multi-zone speech separation method provided by an embodiment of the present invention;

[0018] Figure 8 A four-zone RIR region division diagram of a specific example of a vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention;

[0019] Figure 9 A block diagram of an NN module fused with BF, which is a specific example of a vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention;

[0020] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0022] Please refer to Figure 1 , which shows a flow chart of a vehicle-mounted multi-zone speech separation method provided by an embodiment of the present invention.

[0023] like Figure 1As shown, in step 101, the acquired high-fidelity audio is convolved with the acquired room impulse response data to obtain a mixed signal and at least one speech tag;

[0024] In step 102, a network model of fusion beamforming is trained based on the mixed signal and the at least one speech tag;

[0025] In step 103, the network model of the fused beamforming is tested based on a preset simulation test set to determine whether the network model of the fused beamforming meets preset requirements;

[0026] In step 104, if the preset requirement is met, the beamforming weights of the mixed signal and the at least one voice tag are predicted based on the network model of the fusion beamforming to obtain each voice zone separation signal.

[0027] In this embodiment, for step 101, the vehicle-mounted multi-zone speech separation device convolves the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech label. For example, the mixed signal is randomly selected from a plurality of pre-generated batches of room acoustic impulse response data. The selected room acoustic impulse response data is then convolved with the noise and human voice data in the high-fidelity audio. The data is then mixed based on the actual signal-to-noise ratio and signal-to-interference ratio in the vehicle. Finally, the labeled speech of each zone and the mixed speech are dynamically output.

[0028] Then, for step 102, the vehicle-mounted multi-zone speech separation device trains a fusion beamforming network model based on the mixed signal and at least one voice label. For example, the obtained mixed signal and voice label are transmitted together to the fusion beamforming neural network model for training. The fusion beamforming network model mainly includes a classic Encoder / Decoder structure for up and down sampling to obtain spatial and frequency band features, and LSTM (Long Short-Term Memory) is used in the middle for time dimension modeling. Finally, the beamforming weight is predicted to obtain the separation signal of each sound zone. Furthermore, a skip connection is used between the Encoder and the Decoder to prevent degradation problems.

[0029] Then, for step 103, the vehicle-mounted multi-zone speech separation device tests the network model of the fusion beamforming based on a preset simulation test set to determine whether the network model of the fusion beamforming meets the preset requirements. For example, the obtained network model of the fusion beamforming is used for a preset simulation test to perform a series of tests to detect data to determine whether the network model of the fusion beamforming meets the requirements.

[0030] Finally, for step 104, if the network model of the fusion beamforming meets the preset requirements, the beamforming weights of the mixed signal and at least one voice tag are predicted based on the network model of the fusion beamforming to obtain the separation signal of each sound zone. For example, after the network model of the fusion beamforming meets the requirements, the beamforming weight ratio of the previously input mixed signal and the voice tag is predicted based on this model, and the obtained weight ratio is used to predict the separation signal of each sound zone. If the network model of the fusion beamforming does not meet the preset requirements, it is necessary to adjust the room impulse response parameters and training parameters for retraining.

[0031] The method of this embodiment is to convolve high-fidelity audio with room impulse response data, and then use the data obtained by convolution to train a network model for fusion beamforming. After the training is completed, a preset simulation test is used to test whether the requirements are met. Finally, the network model that meets the requirements is used to predict the weights of the data obtained by convolution, thereby achieving higher resolution and accuracy in sound zone separation.

[0032] Please refer to Figure 2 , which shows a flow chart of another vehicle-mounted multi-zone speech separation method provided by an embodiment of the present invention. The flow chart mainly describes the process Figure 1 A flowchart of the steps further defined in the process after "If the preset requirements are met, the beamforming weights of the mixed signal and the at least one voice tag are predicted based on the network model of the fusion beamforming to obtain the separation signal of each voice zone."

[0033] like Figure 2 As shown, in step 201, the network model based on the fusion beamforming of the separated signals of each sound zone guides the traditional blind source separation algorithm to perform separation and obtain a separation result;

[0034] In step 202, a preset real vehicle test set is collected to verify whether the separation result meets the index requirements;

[0035] In step 203, if the index requirements are met, the multi-zone separated audio is output for use in the subsequent voice interaction system.

[0036] In this embodiment, for step 201, the vehicle-mounted multi-zone speech separation device guides the traditional blind source separation algorithm to perform separation based on the network model of beamforming of the separated signals of each zone to obtain the separation result. In a specific embodiment, since the neural network is modeled by a large number of in-vehicle acoustic environments, it can perform sound zone separation more accurately, so the audio output by the neural network is used to guide the separation of the traditional blind source separation algorithm.

[0037] Then, for step 202, the vehicle-mounted multi-zone speech separation device collects and verifies whether the separation result meets the index requirements based on a preset real vehicle test set.

[0038] Finally, for step 203, if the index requirements are met, the multi-zone separated audio is output for use in the subsequent voice interaction system. Furthermore, if the index requirements are not met, the traditional blind source separation algorithm is continued to be used for separation.

[0039] The method of this embodiment guides the traditional blind source separation algorithm to perform audio separation operations by fusing a beamforming network model, thereby solving the problem of audio distortion and further separating residual interference to achieve better separation effect.

[0040] Please refer to Figure 3 , which shows a flow chart of another vehicle-mounted multi-zone speech separation method provided by an embodiment of the present invention. The flow chart mainly describes the process Figure 1 A flowchart of the steps further defined in the process before “convolving the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech tag”.

[0041] like Figure 3 As shown, in step 301, the three-dimensional coordinate information of the microphone of the vehicle, the location information of the sound source and the interior dimensions of the vehicle are collected;

[0042] In step 302, a multi-zone room impulse response simulation is performed based on the microphone three-dimensional coordinate information, the sound source position information, and the interior dimensions of the vehicle;

[0043] In step 303, random simulation sampling is performed on the sound source positions in the room impulse response simulation area to generate batch room impulse response data.

[0044] In this embodiment, for step 301, the vehicle-mounted multi-zone speech separation device collects the vehicle's microphone three-dimensional coordinate information, sound source position information and vehicle interior dimensions. For example, a preset three-dimensional modeling software is used to collect the microphone position coordinates, sound source position information, the length, width and height of the vehicle interior space and the horizontal distance between the seat headrest and the microphone.

[0045] Then, for step 302 , the vehicle-mounted multi-zone speech separation device performs a multi-zone room impulse response simulation based on the microphone three-dimensional coordinate information, the sound source position information, and the vehicle interior dimensions.

[0046] Finally, for step 303, the vehicle-mounted multi-zone speech separation device performs random simulation sampling of the sound source positions in the room impulse response simulation area to generate batch room impulse response data. For example, a car model is modeled using three-dimensional modeling software to obtain actual vehicle information (microphone three-dimensional coordinate information, sound source position information, and vehicle interior dimensions). Then, based on the obtained vehicle information, a multi-zone room impulse response simulation is performed. Finally, for the multi-zone room impulse response simulation, random sampling is performed on the sound source positions in the simulation area to generate batch room impulse response data. The method of this embodiment collects vehicle information using three-dimensional modeling software and performs multi-zone room impulse response simulation based on the collected information, and samples the sound sources in the simulation area to generate room impulse response data. This allows the obtained data to be more accurate than actual vehicle measurement data, and allows model training to be performed in advance, greatly reducing the overall solution output time.

[0047] In some optional embodiments, the high-fidelity audio data includes:

[0048] Noise collected from real vehicles and high-fidelity voices recorded by a large number of different people. For example, high-fidelity audio is mainly divided into two parts. One part is the noise collected from real vehicles, such as wind noise, tire noise, and air conditioning noise; the other part is high-fidelity voices recorded by a large number of different people.

[0049] In some optional embodiments, the multi-tone area includes at least a dual-tone area, a quad-tone area, a five-tone area, etc.

[0050] In some optional embodiments, the network model of the fused beamforming includes an encoder, a long short-term memory network and a decoder, wherein the encoder and decoder perform up and down sampling to obtain spatial and frequency band features, and the long short-term memory network is used in the middle to perform time dimension modeling, wherein the long short-term memory network (LSTM) is a time recurrent neural network specially designed to solve the long-term dependency problem existing in general RNN (recurrent neural network), and all RNNs have a chain form of repeated neural network modules.

[0051] Please refer to Figure 4 , which shows a flowchart of another method for identifying clue labels for sales calls provided by an embodiment of the present invention. The flowchart mainly describes the process Figure 1 A flowchart of the steps further defined in the process of "convolving the obtained high-fidelity audio with the obtained room impulse response data to obtain a mixed signal and at least one speech tag".

[0052] like Figure 4 As shown, in step 401, a convolution result of the high-fidelity audio and room impulse response data is obtained;

[0053] In step 402, the convolution result is mixed with the actual signal-to-noise ratio and signal-to-interference ratio in the vehicle to obtain mixed audio and at least one voice tag.

[0054] In this embodiment, in step 401, the convolution result of the high-fidelity audio and the room impulse response data is obtained. Then, in step 402, the convolution result is mixed with the actual signal-to-noise ratio and signal-to-interference ratio in the vehicle to obtain mixed audio and at least one voice tag. For example, this is randomly selected from a batch of generated room impulse response data, and the selected room impulse response data is convolved with the noise and human voice data in the high-fidelity audio. The data is then mixed based on the actual signal-to-noise ratio and signal-to-interference ratio in the vehicle. Finally, the labeled voices for each voice zone and the mixed voice are dynamically output.

[0055] The method of this embodiment obtains the convolution result of the high-fidelity audio and the room impulse response data, and then mixes the convolution result with the actual signal-to-noise ratio and signal-to-interference ratio in the car, thereby improving the resolution and accuracy of sound zone separation.

[0056] In some optional embodiments, after determining whether the network model of the fused beamforming meets preset requirements, the method further includes:

[0057] If the requirements are not met, readjust the room acoustic impulse response and training parameters and then retrain.

[0058] Please refer to Figure 5 , which shows a structural diagram of a dual-tone zone of a specific example of a vehicle-mounted multi-tone zone speech separation method provided by an embodiment of the present invention.

[0059] like Figure 5 As shown in the figure, audio zone 1 represents the driver's seat, and audio zone 2 represents the passenger seat. When the driver's mouth is located in the fuzzy area, such as the red five-pointed star position, the algorithm will assign the separated audio to the passenger seat, thus affecting the subsequent user interaction experience. Figure 6 , which shows a design block diagram of vehicle-mounted multi-zone speech separation, which is a specific example of a vehicle-mounted multi-zone speech separation method provided by an embodiment of the present invention.

[0060] like Figure 6 As shown, real-car information collection: Using the car manufacturer's 3D modeling software, we acquire interior spatial information, such as the microphone's position coordinates, the interior dimensions of the space, and the horizontal distance between the headrest and the microphone. This in-car spatial information collection allows us to acquire prior knowledge of the actual vehicle's acoustic environment for subsequent modeling.

[0061] High-fidelity audio input: High-fidelity audio is mainly divided into two parts. One part is noise collected from real vehicles, such as wind noise, tire noise, and air conditioning noise. The other part is high-fidelity voices recorded by a large number of different people.

[0062] Mixed audio input: In vehicle scenarios, local playback and TTS announcements are present, so the input audio includes a reference loop for AEC echo cancellation. Both traditional and NN solutions rely on the AEC module for preprocessing. Since the AEC module is not the focus of this patent, the default mixed audio input has already been echo-cancelled by the AEC module.

[0063] RIR Simulation Module: The room impulse response (RIR) is primarily based on the image method proposed by Allen and Berkley in 1979. This method is widely used in acoustic signal processing. This patent primarily uses the image method to simulate in-vehicle RIR.

[0064] Training Data Generation Module: During the training phase, a randomly selected RIR is convolved with the noise and human voice data in the high-fidelity audio. The RIR is then mixed based on the actual signal-to-noise ratio and signal-to-interference ratio in the vehicle. Finally, the labeled (1abel) speech for each voice region and the mixed speech are dynamically output for network training.

[0065] NN module integrated with BF: In order to solve the problem of combining NN with traditional BF algorithms, the traditional solution stage is mainly based on statistical theory, and the NN performance is limited, resulting in a large fuzzy boundary. This patent proposes a NN module integrated with BF, which extracts and processes the features of the original input signal through a large amount of prior knowledge, and outputs the separation signal of each sound zone. On the one hand, the network training of the acoustic environment in the car greatly reduces the size of the fuzzy interval; on the other hand, the use of the NN module integrated with BF solves the dependence of the traditional BF algorithm on the microphone spacing, which is conducive to the large-scale implementation of multi-sound zone separation in the vehicle.

[0066] Traditional algorithm modules: Because simulated RIRs still differ somewhat from actual ones, audio distortion and residual non-target paths can occur. To address this issue, this paper improves the traditional BSS algorithm based on the audio output of the NN module. Since NN has smaller fuzzy boundaries, it achieves higher separation accuracy than traditional algorithms, which offer greater robustness and less distortion. Therefore, using the audio output of the NN module to guide the traditional BSS algorithm's separation will further optimize the separation effect.

[0067] Please refer to Figure 7, which shows a flowchart of a vehicle-mounted multi-zone speech separation process of a specific example of a vehicle-mounted multi-zone speech separation method provided by an embodiment of the present invention.

[0068] like Figure 7 As shown, step 1: real vehicle information collection, collecting microphone three-dimensional coordinate information, sound source location information and vehicle interior dimensions, such as length, width and height.

[0069] Step 2: Real vehicle room impulse simulation, using the relative position information of the microphone and sound source collected in step 1 and the interior space of the vehicle to simulate. By randomly sampling the sound source positions in the area, batch room impulse responses are generated for subsequent training.

[0070] Step 3: Generate training data. The training process inputs randomly extracted data from the high-fidelity audio convolution and mixes it in real time according to different signal-to-noise ratios and signal-to-interference ratios. The resulting mixed signal and the label signals of each sound range (i.e., audio in different sound ranges) are then fed into the network model training.

[0071] Step 4: Network training: The mixed signal obtained in the previous step and the label signals of each sound zone are used to train the fusion beamforming network model. The trained model needs to be tested using a simulation test set to see if it meets the requirements. If it does not meet the requirements, it is necessary to return to step 2 to readjust its parameters and subsequent training parameters and retrain. If it meets the requirements, it will be used as the subsequent fusion beamforming network model for subsequent separation tasks.

[0072] Step 5: The neural network module integrated with beamforming is used to perform multi-zone speech separation. The encoder / decoder structure is used for up-sampling and down-sampling to obtain spatial and frequency band features. Long-short-term memory is used for temporal dimension modeling in the middle, and the beamforming weights are predicted to obtain the separated signals of each zone.

[0073] Step 6: Traditional algorithm. Since the neural network is modeled by a large number of in-vehicle acoustic environments, it can perform speech separation more accurately. Therefore, the audio output by the neural network is used to guide the separation of the traditional blind source separation algorithm, and the actual vehicle test set is used to verify whether the requirements are met. If the requirements are not met, the parameters of the blind source separation need to be readjusted. If the requirements are met, the multi-zone speech separation audio is output for subsequent voice interaction systems, etc.

[0074] Please refer to Figure 8 , which shows a four-zone RIR area division diagram of a specific example of a vehicle-mounted multi-zone speech separation method provided by an embodiment of the present invention.

[0075] like Figure 8 As shown in the figure, the blue area is the actual RIR simulation area. By randomly sampling the sound source positions in the area, batch RIRs are generated for the subsequent training process.

[0076] Figure 9 A block diagram of an NN module fused with BF, which is a specific example of a vehicle-mounted multi-zone speech separation method provided by one embodiment of the present invention.

[0077] like Figure 9 As shown, the encoder and decoder structures are used for up- and down-sampling to obtain spatial and frequency band features; LSTM (Long Short-Term Memory) is used between the encoder and decoder structures to obtain modeling of the time dimension; prediction is made based on the obtained spatial and frequency band features and the modeling of the time dimension to predict the BF (beamforming) weight; the separation signal of each sound zone is obtained based on the BF weight; and skip links are used between the encoder and decoder to prevent degradation problems.

[0078] In some optional embodiments, for NN networks fused with BF, traditional convolutions in the encoder / decoder and LSTM are replaced with complex convolutional networks. By using two-dimensional convolutions to model the real and imaginary parts of the input multi-channel signal, the network's feature extraction capabilities are greatly improved, but this also increases the network's computing power. To balance computing power and separation performance, complex convolutions were ultimately not used for network construction.

[0079] In some optional embodiments, a combination of NN and GSC is employed. The NN algorithm first outputs the separated audio and mask, which are then fed into an adapted GSC algorithm to assist in calculating the fixed weights W and the blocking matrix B. Finally, the GSC outputs the separated signal. This allows for better separation performance and robustness for fine-pitch arrays.

[0080] In other embodiments, embodiments of the present invention further provide a non-volatile computer storage medium storing computer executable instructions, the computer executable instructions being capable of executing the lead tag identification method for sales calls in any of the above method embodiments;

[0081] As an embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows:

[0082] Convolving the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech label;

[0083] Training a network model of fusion beamforming based on the mixed signal and the at least one voice tag;

[0084] Testing the network model of the fused beamforming based on a preset simulation test set to determine whether the network model of the fused beamforming meets preset requirements;

[0085] If the preset requirements are met, the beamforming weights of the mixed signal and the at least one voice tag are predicted based on the network model of the fusion beamforming to obtain each voice zone separation signal.

[0086] The non-volatile computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the lead tag identification device for sales calls, etc. In addition, the non-volatile computer-readable storage medium may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the lead tag identification device for sales calls via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0087] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any one of the above-mentioned lead tag identification methods for sales calls.

[0088] Figure 10 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 10 As shown, the device includes: one or more processors 1010 and a memory 1020, Figure 10 A processor 1010 is used as an example. The device for the method for identifying lead tags of sales calls may further include: an input device 1030 and an output device 1040. The processor 1010, the memory 1020, the input device 1030 and the output device 1040 may be connected via a bus or other means. Figure 10The example of the bus connection is taken. The memory 1020 is the non-volatile computer-readable storage medium mentioned above. The processor 1010 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 1020, that is, implements the clue tag identification method for sales calls in the above method embodiment. The input device 1030 can receive input digital or character information, and generate key signal input related to the user settings and function control of the clue tag identification device for sales calls in the embodiment. The output device 1040 may include a display device such as a display screen.

[0089] The above-mentioned product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0090] As an embodiment, the electronic device is applied to a lead tag identification device for a sales call, and includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0091] Convolving the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech label;

[0092] Training a network model of fusion beamforming based on the mixed signal and the at least one voice tag;

[0093] Testing the network model of the fused beamforming based on a preset simulation test set to determine whether the network model of the fused beamforming meets preset requirements;

[0094] If the preset requirements are met, the beamforming weights of the mixed signal and the at least one voice tag are predicted based on the network model of the fusion beamforming to obtain each voice zone separation signal.

[0095] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to:

[0096] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communications. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0097] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0098] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0099] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0100] (5) Other electronic devices with data interaction functions.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. That is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A vehicle-mounted multi-zone speech separation method, comprising: Convolving the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech label; Training a network model of fusion beamforming based on the mixed signal and the at least one voice tag; Testing the network model of the fused beamforming based on a preset simulation test set to determine whether the network model of the fused beamforming meets preset requirements; If the preset requirements are met, the beamforming weights of the mixed signal and the at least one voice tag are predicted based on the network model of the fusion beamforming to obtain each voice zone separation signal.

2. The method according to claim 1, wherein After the preset requirement is met, the method further includes predicting beamforming weights of the mixed signal and the at least one voice tag based on the fusion beamforming network model to obtain each voice zone separation signal; Based on the separation signal of each sound zone, a traditional blind source separation algorithm is guided to perform separation to obtain a separation result; Verify whether the separation results meet the index requirements based on a preset real vehicle test set; If the index requirements are met, multi-zone separated audio will be output for subsequent voice interaction system.

3. The method according to claim 1, wherein Before convolving the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech tag, the method further includes: Collect the vehicle's microphone three-dimensional coordinate information, sound source location information and vehicle interior dimensions; Performing a multi-zone room impulse response simulation based on the microphone three-dimensional coordinate information, the sound source location information, and the interior dimensions of the vehicle; Randomly simulate and sample the sound source positions in the room impulse response simulation area to generate batch room impulse response data.

4. The method according to claim 1, wherein The high-fidelity audio data includes: Noise collected from real vehicles and high-fidelity voices recorded by a large number of different people.

5. The method according to claim 2, wherein the multi-tone zone comprises at least: Two-tone area, four-tone area and five-tone area.

6. The method according to claim 1, wherein The network model of the fusion beamforming includes an encoder, a long short-term memory network and a decoder, wherein the encoder and decoder perform up and down sampling to obtain spatial and frequency band features, and the long short-term memory network is used in the middle to perform time dimension modeling.

7. The method according to claim 1, wherein The convolving the acquired high-fidelity audio with the acquired room impulse response data to obtain a mixed signal and at least one speech tag includes: Obtaining a convolution result of the high-fidelity audio and the room impulse response data; The convolution result is mixed with an actual signal-to-noise ratio and a signal-to-interference ratio in the vehicle to obtain mixed audio and at least one voice tag.

8. The method according to claim 1, wherein After determining whether the network model of the fused beamforming meets the preset requirements, the method further includes: If the requirements are not met, readjust the room impulse response and training parameters and then retrain.

9. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images