Real-time call voiceprint noise reduction method, electronic device and storage medium

Abstract

The real-time call voiceprint noise reduction method, the electronic device and the storage medium, wherein the real-time call voiceprint noise reduction method comprises: acquiring real-time call audio, a first voiceprint feature vector of a current speaker and registered audio of the current speaker; inputting the registered audio into a voiceprint feature extraction network trained synchronously with a pre-trained noise reduction network to obtain a second voiceprint feature vector; inputting the real-time call audio into an encoding part of the pre-trained noise reduction network to obtain a third voiceprint feature vector, wherein the pre-trained noise reduction network comprises the encoding part and other parts; and inputting the first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector after splicing into the other parts of the pre-trained noise reduction network. Thus, the output audio can better retain the audio of the speaker and effectively suppress the interference of other voices.

Description

Technical Field

[0001] This application relates to the field of voice processing technology, and in particular to a real-time call voiceprint noise reduction method, electronic device, and storage medium. Background Technology

[0002] Voiceprint noise reduction, as the name suggests, adds voiceprint information to call noise reduction technology. Even in noisy environments or complex scenarios with multiple people speaking, it can extract clear voice information of the main speaker while filtering out the voices of other speakers and background noise. This technology has a very wide range of applications in practical production and daily life.

[0003] In existing technologies, voiceprint noise reduction solutions include a registration phase and a testing phase.

[0004] The registration phase involves the speaker registering a 20-30 second audio clip in a quiet environment, following prompts on the user interface (UI), to extract their voiceprint information. In real-world use, unclear speech, high background noise, fast speaking speed, and short recording times can all affect information collection and consequently the algorithm's performance. Therefore, the registration phase imposes restrictions on audio quality. Specifically, speech quality is assessed using Voice Activity Detection (VAD), and the signal-to-noise ratio (SNR) is calculated based on the audio and background noise. Word accuracy is verified by comparing the recognized text with the correct text after speech recognition. Finally, the VAD-generated audio must meet a certain length requirement. Only after these conditions are met can voiceprint information be correctly registered.

[0005] Testing Phase: Since most PC products on the market currently use more than one microphone, microphone arrays composed of multiple microphones can better personalize and enhance the voice signal. Voiceprint noise reduction generally requires enhancing the voice of the speaker directly in front of the laptop while suppressing voice signals from other directions, as well as suppressing other interfering voices from the same direction. This involves using the following technologies: echo cancellation, microphone array technology, voiceprint noise reduction technology, and automatic gain control technology.

[0006] Currently, the available devices for testing, besides laptops and mobile phones, are limited in variety and have poor performance. Their main shortcomings include: ineffective suppression of the registered speaker's voice, with the main speaker's voice fluctuating in volume and even omitting words. This is primarily because the model confuses the main speaker's voice with those of interfering speakers, suppressing both; and incomplete suppression of interfering voices, resulting in audio remnants or even failing to eliminate them entirely. However, based on our offline audio processing results, we found that our algorithm can effectively preserve the registered speaker's audio and suppress interference from other speakers.

[0007] Voiceprint noise reduction, like other noise reduction methods, also faces the problem of either not completely eliminating the speaker's voice or eliminating it excessively. However, in actual use, we will try our best to preserve the speaker's voice as much as possible, even if some interfering voice remains. Summary of the Invention

[0008] This invention provides a method and apparatus for real-time voiceprint noise reduction in voice calls, which at least solves one of the above-mentioned technical problems.

[0009] In a first aspect, embodiments of the present invention provide a real-time call voiceprint noise reduction method, comprising: acquiring real-time call audio, a first voiceprint feature vector of the current speaker, and a registered audio of the current speaker; inputting the registered audio into a voiceprint feature extraction network trained synchronously with a pre-trained noise reduction network to obtain a second voiceprint feature vector; inputting the real-time call audio into the encoding part of the pre-trained noise reduction network to obtain a third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts; and concatenating the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector and inputting the concatenated vector into other parts of the pre-trained noise reduction network.

[0010] In a second aspect, embodiments of the present invention provide an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described real-time call voiceprint noise reduction methods of the present invention.

[0011] Thirdly, embodiments of the present invention provide a storage medium storing one or more programs including execution instructions, the execution instructions being readable and executed by electronic devices (including but not limited to computers, servers, or network devices, etc.) to perform any of the above-described real-time call voiceprint noise reduction methods of the present invention.

[0012] Fourthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-mentioned real-time call voiceprint noise reduction methods.

[0013] The method of this application acquires real-time call audio, the first voiceprint feature vector of the current speaker, and the registered audio of the current speaker. Then, the registered audio is input into a synchronously trained voiceprint feature extraction network to obtain the second voiceprint feature vector. The real-time call audio is then input into the encoder part of a pre-trained noise reduction network to obtain the third audio feature vector. Finally, the first voiceprint feature vector, the second voiceprint feature vector, and the third audio feature vector are concatenated and output to other parts of the pre-trained noise reduction network for processing. This allows the output audio to better preserve the speaker's audio and effectively suppress interference from other voices. Attached Figure Description

[0014] Figure 1 A flowchart of a real-time call voiceprint noise reduction method provided in an embodiment of the present invention;

[0015] Figure 2 A flowchart of another real-time call voiceprint noise reduction method provided in an embodiment of the present invention;

[0016] Figure 3 A schematic diagram of speaker over-elimination, which is a specific example of the prior art provided for an embodiment of the present invention;

[0017] Figure 4 A schematic diagram illustrating that the interference was not completely eliminated, as a specific example of the prior art provided for an embodiment of the present invention;

[0018] Figure 5 A network framework diagram for real-time call voiceprint noise reduction, which is a specific example of a real-time call voiceprint noise reduction method provided in an embodiment of the present invention.

[0019] Figure 6 A flowchart of the real-time call voiceprint noise reduction registration stage, which is a specific example of a real-time call voiceprint noise reduction method provided in an embodiment of the present invention.

[0020] Figure 7 A flowchart of the real-time call voiceprint noise reduction test phase, which is a specific example of a real-time call voiceprint noise reduction method provided in an embodiment of the present invention.

[0021] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please refer to Figure 1 The diagram shows a flowchart of a real-time call voiceprint noise reduction method provided by an embodiment of the present invention.

[0024] like Figure 1 As shown, in step 101, the real-time call audio, the first voiceprint feature vector of the current speaker, and the registered audio of the current speaker are obtained;

[0025] In step 102, the registered audio is input into a voiceprint feature extraction network that is trained synchronously with a pre-trained noise reduction network to obtain a second voiceprint feature vector;

[0026] In step 103, the real-time call audio is input to the encoding part of the pre-trained noise reduction network to obtain the third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts;

[0027] In step 104, the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector are concatenated and input into other parts of the pre-trained noise reduction network.

[0028] In this embodiment, for step 101, the real-time call audio, the first voiceprint feature vector of the current speaker, and the registered audio of the current speaker are obtained. For example, during a call, the real-time call audio is obtained through the device, and the first voiceprint feature vector and the registered audio are obtained in advance. Furthermore, the obtained first voiceprint feature vector and the registered audio belong to the same speaker.

[0029] Then, for step 102, the registered audio is input into the voiceprint feature extraction network trained synchronously with the pre-trained noise reduction network to obtain the second voiceprint feature vector. For example, the voiceprint information of the registered audio is extracted by the voiceprint feature extraction network trained synchronously with the pre-trained noise reduction network to obtain the second voiceprint feature vector.

[0030] Then, for step 103, the real-time call audio is input into the encoding part of the pre-trained noise reduction network to obtain the third voiceprint feature vector. The pre-trained noise reduction network includes an encoding part and other parts. For example, the real-time call audio obtained by the device is input into the pre-trained noise reduction network to obtain the third voiceprint feature vector through the encoder. The pre-trained noise reduction network includes an encoding part and other parts.

[0031] Finally, for step 104, the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector are concatenated and then input into other parts of the pre-trained noise reduction network. For example, the first, second, and third voiceprint feature vectors are concatenated, and then the concatenated voiceprint feature vector is input into other parts of the pre-trained noise reduction network for processing.

[0032] The method in this embodiment is to acquire real-time call audio, the first voiceprint feature vector of the current speaker, and the registered audio of the current speaker. Then, the registered audio is input into a voiceprint feature extraction network that is trained simultaneously to obtain the second voiceprint feature vector. The real-time call audio is then input into the encoder part of a pre-trained noise reduction network to obtain the third audio feature vector. Finally, the first voiceprint feature vector, the second voiceprint feature vector, and the third audio feature vector are concatenated and output to other parts of the pre-trained noise reduction network for processing. This allows the output audio to better preserve the speaker's audio and effectively suppress interference from other voices.

[0033] In some optional embodiments, the first voiceprint feature vector of the current speaker is obtained by inputting the registered audio of the current speaker into a fixed speaker feature extractor for feature extraction, thereby obtaining the first voiceprint feature vector in advance to facilitate subsequent operations.

[0034] Please refer to the following for further information. Figure 2 The document illustrates a flowchart of another real-time call voiceprint noise reduction method according to an embodiment of the present invention. This flowchart primarily describes the process... Figure 1 A further step diagram defining the training steps of "synchronous training of the pre-trained noise reduction network and the voiceprint feature extraction network".

[0035] like Figure 2 As shown, in step 201, the noisy frequency is input into the encoding part of the pre-trained noise reduction network to obtain the encoded result, wherein the noisy frequency has a corresponding clean audio and a registered audio belonging to the same speaker as the noisy frequency;

[0036] In step 202, at least the voiceprint extraction result obtained by the voiceprint feature extraction network of the registered audio is concatenated with the encoded result to obtain the concatenated result;

[0037] In step 203, the spliced ​​result is further input into other parts of the pre-trained noise reduction network for processing to obtain the output of the pre-trained noise reduction network;

[0038] In step 204, the output of the pre-trained noise reduction network and the loss of the clean audio are calculated, and the pre-trained noise reduction network and the voiceprint extraction model are trained based on the loss.

[0039] In this embodiment, for step 201, the noisy frequency is input into the encoding part of the pre-trained noise reduction network to obtain the encoded result. The noisy frequency has a corresponding clean audio and a registered audio belonging to the same speaker as the noisy frequency. For example, during the training phase, the noisy frequency prepared in advance is input into the pre-trained noise reduction network and processed by its encoder to obtain the encoded result. The noisy frequency has a corresponding clean audio and a registered audio belonging to the same speaker as the noisy frequency.

[0040] Then, for step 202, at least the voiceprint extraction result obtained by the voiceprint feature extraction network of the registered audio is concatenated with the encoded result to obtain the concatenated result. For example, the registered audio needs to obtain the voiceprint extraction result through the voiceprint network, and then the voiceprint extraction result is concatenated with the encoded result to obtain the concatenated result.

[0041] Then, for step 203, the spliced ​​result is further input into other parts of the pre-trained noise reduction network for processing to obtain the output of the pre-trained noise reduction network. For example, after splicing the voiceprint extraction result with the encoded result, it is input into other parts of the pre-trained noise reduction network for processing and output to obtain its output audio.

[0042] Finally, for step 204, the output of the pre-trained noise reduction network and the loss of the clean audio are calculated. The pre-trained noise reduction network and the voiceprint extraction model are trained based on the loss. For example, the audio output of the pre-trained noise reduction network is compared with the clean audio to obtain the loss of the processed audio. The pre-trained noise reduction network and the voiceprint extraction model are trained based on the audio loss.

[0043] The method in this embodiment inputs the noisy frequencies of the corresponding clean audio and the registered audio of the same speaker into the encoder part of the pre-trained voiceprint feature extraction network to obtain the encoded result. Then, the registered audio is input into the voiceprint extractor to obtain the extracted result. After obtaining the voiceprint extracted result, it is concatenated with the encoded result and input into other parts of the pre-trained denoising network for processing to obtain the output audio. Finally, the audio output by the pre-trained denoising network is compared with the clean audio to obtain the loss of the processed audio. The pre-trained denoising network and the voiceprint extraction model are trained by the audio loss.

[0044] In some possible embodiments, the registered audio can first be input into a fixed speaker feature vector extractor to obtain a fixed extraction result, then the registered audio can be input into the voiceprint feature extraction network to obtain the voiceprint extraction result, and finally the fixed extraction result, the voiceprint feature extraction network extraction result, and the encoded result can be concatenated to obtain the concatenated result, thereby making the concatenated audio more accurate.

[0045] In some alternative embodiments, the output of the pre-trained denoising network and the loss of clean audio are calculated using the scale-invariant source-to-noise ratio (SISNR) loss function. The reason for using the SISNR loss function is that no speaker-related loss function is added during the training phase of the model. This allows the pre-trained denoising network to only have the function of eliminating interfering human voices. Then, the low-dimensional voiceprint information is fully utilized through the SISNR loss function, thereby achieving the function of eliminating irrelevant voiceprint information and retaining only the voiceprint information of the registered speaker.

[0046] In some alternative embodiments, the noisy frequency is obtained by mixing the clean audio of the current speaker with the audio of at least one interfering party; the mixing can be done directly. Alternatively, the clean audio can be first mixed by superimposing different room impulse responses, and then different signal-to-noise ratio noises can be added to the speaker and the interfering party, so that the noisy frequency can better simulate real-world call scenarios.

[0047] In some optional embodiments, the real-time call audio needs to be processed by acoustic echo cancellation (AEC) and beamforming (BF) before being input into the encoding part of the pre-trained noise reduction network. The processed audio is then input into the pre-trained noise reduction network for automatic gain control (AGC), which makes the final output audio sound more comfortable and smooth.

[0048] In some optional embodiments, the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector are concatenated and then processed by other parts of the pre-trained noise reduction network, so that the processed audio can be sent to a remote person in a real-time call or input into a speech recognition engine.

[0049] Please refer to Figure 3 This illustrates a schematic diagram of speaker over-elimination, a specific example of the prior art provided in an embodiment of the present invention.

[0050] like Figure 3 As shown, the Mix audio is the noisy frequency, and the Ref audio is the clean audio corresponding to the noisy frequency. The denoised audio shows that the noisy frequency of the registered speaker is over-cancelled after being processed by the denoising network. This is because the denoising network confuses the registered speaker's voice with the voice of the interfering person.

[0051] Please refer to Figure 4 This illustration shows a schematic diagram of a specific example of the prior art provided by an embodiment of the present invention where the interference has not been completely eliminated.

[0052] like Figure 4 As shown, the Mix audio is the noisy frequency, and the Ref audio is the clean audio corresponding to the noisy frequency. The denoised audio shows that the noisy frequency of the registered speaker is not completely suppressed by the denoising network, resulting in noise residue.

[0053] In implementing this application, the inventors attempted the following technical solutions: One solution used a fixed speaker vector, without updating this module during model training. The result of this solution was insufficient removal of other people's voices. Another solution involved jointly training the speaker model and the speech enhancement model. This method was more complex during model training and failed to consider real-time performance issues in real-world scenarios.

[0054] The technical solution of this application is designed and optimized from the following aspects:

[0055] The speaker extraction method of this application consists of two parts: one is an embedding module for extracting text-independent speaker recognition features. This module is pre-trained by the model and does not need to be retrained during the training of the speaker noise reduction model.

[0056] Another part of the voiceprint feature extraction changes with the network weights during the training of the voiceprint denoising model. This allows for the extraction of as much information as possible about the speaker, ensuring that the speaker does not miss any words during actual use.

[0057] In the data preparation phase, registered audio recordings for each speaker, each 30 seconds long, are prepared, and test audio recordings for each speaker are concatenated into a single long audio file. During model training, data is read in real-time. Each time, a 10-second quiet audio recording of the main speaker is selected, and 0-3 interfering persons can be selected each time. Noise with different signal-to-noise ratios is added to both the main speaker and the interfering persons. To better simulate real-world scenarios, the clean audio can be overlaid with impact responses from different rooms.

[0058] Please refer to Figure 5 The diagram illustrates a network framework diagram for real-time call voiceprint noise reduction, representing a specific example of a real-time call voiceprint noise reduction method provided in an embodiment of the present invention.

[0059] like Figure 5 As shown, we combine speech enhancement and speaker feature vector extraction models. Step 1: Extract the EMB, where EMB refers to the embedded features mentioned earlier. The prepared data, including registered audio, clean audio, and mixed audio, are all converted to the frequency domain. Here, we use a Hanning window, a frame length of 512, a frame shift of 256, and an FFT length of 512.

[0060] Step 2: Perform the noise reduction network operation on the right side of the figure for the noisy audio, and calculate the SISNR loss of the network output and the clean audio.

[0061] Step 3: After feature extraction, the registered audio is input into the network and further processed in the time dimension. Then, the same operation is performed on the noisy audio. The output of the registered audio is concatenated with the noisy audio after inputting into the network. Here, I-Feature refers to converting the speech from the frequency domain to the time domain.

[0062] Please refer to Figure 6 The diagram illustrates a flowchart of the real-time call voiceprint noise reduction registration stage, representing a specific example of a real-time call voiceprint noise reduction method provided in an embodiment of the present invention.

[0063] like Figure 6 As shown, Step 1: The user pre-registers with a registration audio of about 20-30 seconds; Step 2: The registration audio is quality checked; Step 3: The speaker's voiceprint feature 1 is extracted; Step 4: The speaker's voiceprint feature 2 is extracted. Among them, voiceprint feature 1 is a fixed feature extracted by a fixed speaker feature extractor, and voiceprint feature 2 is extracted by a voiceprint feature extraction network that is synchronously trained by a pre-trained noise reduction network.

[0064] Please refer to Figure 7 The diagram illustrates a flowchart of the real-time call voiceprint noise reduction test phase, representing a specific example of a real-time call voiceprint noise reduction method provided in an embodiment of the present invention.

[0065] like Figure 7 As shown, Step 1: The user selects the voice separation mode. Step 2: The microphone records the mixed audio; Step 3: The mixed voice is acquired and its voiceprint features are extracted; Step 4: The audio is processed using Acoustic Echo Cancellation (AEC); Step 5: The audio is processed using Beam Forming (BF); Step 6: The audio is input to a voiceprint noise reduction network for processing.

[0066] Step 7: The processed audio input is amplified by Automatic Gain Control (AGC) and then output as audio.

[0067] In other embodiments, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the lead tag identification method for sales telephones in any of the above method embodiments.

[0068] In one embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0069] Obtain the real-time call audio, the first voiceprint feature vector of the current speaker, and the registered audio of the current speaker;

[0070] The registered audio is input into a voiceprint feature extraction network that is trained synchronously with a pre-trained noise reduction network to obtain a second voiceprint feature vector;

[0071] The real-time call audio is input into the encoding part of the pre-trained noise reduction network to obtain the third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts;

[0072] The first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector are concatenated and then input into other parts of the pre-trained noise reduction network.

[0073] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the lead tag identification device for telemarketing calls, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely configured relative to a processor, which can be connected to the lead tag identification device for telemarketing calls via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0074] This invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform any of the above-described methods for real-time voiceprint noise reduction.

[0075] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 8 As shown, the device includes one or more processors 810 and a memory 820. Figure 8 Taking a processor 810 as an example, the device for a lead tagging method for sales telephones may further include an input device 830 and an output device 840. The processor 810, memory 820, input device 830, and output device 840 can be connected via a bus or other means. Figure 8 Taking a bus connection as an example, the memory 820 is the aforementioned non-volatile computer-readable storage medium. The processor 810 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 820, thereby implementing the lead tag identification method for sales telephones described in the above method embodiment. The input device 830 can receive input numeric or character information and generate key signal inputs related to user settings and function control of the lead tag identification device for sales telephones in the embodiment. The output device 840 may include a display screen or other display device.

[0076] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0077] In one embodiment, the above-described electronic device is applied to a lead tag identification device for telephone sales, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0078] Obtain the real-time call audio, the first voiceprint feature vector of the current speaker, and the registered audio of the current speaker;

[0079] The registered audio is input into a voiceprint feature extraction network that is trained synchronously with a pre-trained noise reduction network to obtain a second voiceprint feature vector;

[0080] The real-time call audio is input into the encoding part of the pre-trained noise reduction network to obtain the third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts;

[0081] The first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector are concatenated and then input into other parts of the pre-trained noise reduction network.

[0082] The electronic devices described in this application exist in various forms, including but not limited to:

[0083] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0084] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0085] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (e.g., iPods), handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0086] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0087] (5) Other electronic devices with data interaction functions.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for real-time voiceprint noise reduction, comprising: The system acquires real-time call audio, the first voiceprint feature vector of the current speaker, and the registered audio of the current speaker, wherein the first voiceprint feature vector is obtained by using a fixed speaker feature extractor to extract features from the registered audio of the current speaker. The registered audio is input into a voiceprint feature extraction network that is trained synchronously with a pre-trained noise reduction network to obtain a second voiceprint feature vector; The real-time call audio is input into the encoding part of the pre-trained noise reduction network to obtain the third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts; The first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector are concatenated and then input into other parts of the pre-trained noise reduction network.

2. The method according to claim 1, wherein, The synchronous training method for the pre-trained noise reduction network and the speaker feature extraction network includes: The noisy frequency is input into the encoding part of the pre-trained noise reduction network to obtain the encoded result, wherein the noisy frequency has a corresponding clean audio and a registered audio belonging to the same speaker as the noisy frequency; At least the voiceprint extraction result obtained by the voiceprint feature extraction network of the registered audio is concatenated with the encoded result to obtain the concatenated result; The spliced ​​result is then input into other parts of the pre-trained noise reduction network for processing to obtain the output of the pre-trained noise reduction network. Calculate the output of the pre-trained noise reduction network and the loss of the clean audio, and train the pre-trained noise reduction network and the speaker extraction model based on the loss.

3. The method according to claim 2, wherein, The step of concatenating the voiceprint extraction result obtained by the voiceprint feature extraction network with the encoded result to obtain the concatenated result includes: The fixed extraction result obtained by the fixed speaker feature extractor, the voiceprint extraction result obtained by the voiceprint feature extraction network, and the encoded result are concatenated to obtain the concatenated result.

4. The method according to claim 2 or 3, wherein, The calculation of the output of the pre-trained noise reduction network and the loss of the clean audio includes: Calculate the output of the pre-trained noise reduction network and the scale-invariant signal-to-noise ratio loss of the clean audio.

5. The method according to claim 2, wherein, The noisy frequency is obtained through the following method: The noisy frequency is obtained by mixing the clean audio of the current speaker with the interfering audio of at least one interfering person.

6. The method according to claim 1, wherein, Before inputting the real-time call audio into the encoding portion of the pre-trained noise reduction network, the method further includes: The real-time call audio is subjected to echo cancellation and beamforming processing. After concatenating the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector and inputting them into other parts of the pre-trained noise reduction network, the method further includes: The output of the pre-trained noise reduction network is subjected to automatic gain control processing.

7. The method according to claim 1 or 6, wherein, After concatenating the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector and inputting them into other parts of the pre-trained noise reduction network, the method further includes: Send the output of the pre-trained noise reduction network to the remote person in the real-time call; or The output of the pre-trained noise reduction network is fed into the speech recognition engine.

8. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images