Registration-free voiceprint call noise reduction method, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a registration-free voiceprint call noise reduction method, electronic equipment and a storage medium, and the method comprises the steps: obtaining effective audios meeting a total duration threshold value in a real-time call process, carrying out the clustering of the effective audios, and obtaining at least one type of sample points; the class of sample points with the most sample data in the at least one class of sample points is obtained, the effective audios of the class of sample points with the most sample data are spliced for voiceprint registration, voiceprint information is obtained, and the effective audios of the class of sample points with the most sample data are audios of a call speaker; and assisting a trained noise reduction neural network based on the voiceprint information, so that the noise reduction neural network learns the difference of voiceprint features of each frame in the voiceprint information, and obtaining a denoised speaker audio.

Description

Technical Field

[0001] The present invention belongs to the technical field of call noise reduction, and in particular relates to a registration-free voiceprint call noise reduction method, an electronic device and a storage medium. Background Art

[0002] Related products: Lenovo PC personalized call noise reduction, Huawei PC quiet space, Honor magicPad2 AI voiceprint noise reduction, etc. Related technologies: Deep Attractor Network (DANet), Dual-Path RNN (DPRNN), SpeakerBeam speaker extraction algorithm, VoiceFilter speech separation model, Audio-Visual Speech Enhancement (AVSE).

[0003] For existing technologies or products, clustering registration is performed during a call. In noisy environments or multi-person conferences, it is difficult to obtain a relatively pure speaker's voice through voiceprint clustering. The audio with a low signal-to-noise ratio, the speaker's audio with interference, and the audio of other interfering people are clustered. This will suppress the speaker. At present, real-time voiceprint noise reduction interferes with peers, and the performance is not ideal when multiple people speak at the same time. The algorithm based on the neural network has the problem of noise that cannot be eliminated, and it is impossible to accurately and timely estimate the sudden non-stationary noise, and it is impossible to accurately eliminate the noise that has not been seen in the training set, such as cat and dog barking, cicada noise, etc.

[0004] The inventors found that the above technology clustering is inaccurate, causing other interfering voices to be mistakenly included in the speaker's registration information. In this way, after the voiceprint is registered, it will contain information of multiple speakers. This may affect the speaker's call and reduce the performance of suppressing interfering people. Voiceprint noise reduction itself is difficult, and streaming voiceprint noise reduction has strict requirements on latency, especially when affected by interfering people in the same industry, the performance is generally poor. Non-stationary noise cannot be removed in time. Summary of the invention

[0005] The embodiments of the present invention provide a registration-free voiceprint noise reduction method, an electronic device, and a storage medium, which are used to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a registration-free voiceprint noise reduction method, comprising: obtaining valid audio that meets a total duration threshold during a real-time call, clustering the valid audio to obtain at least one type of sample points; obtaining a type of sample points with the most sample data among the at least one type of sample points, and splicing the valid audio of the type of sample points with the most sample data to perform voiceprint registration to obtain voiceprint information, wherein the valid audio of the type of sample points with the most sample data is the audio of the speaker of the call; based on the voiceprint information, assisting a trained denoising neural network to enable the denoising neural network to learn the differences in voiceprint features of each frame in the voiceprint information to obtain the denoised audio of the speaker.

[0007] In a second aspect, an embodiment of the present invention further provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the method described in the first aspect.

[0008] In a third aspect, an embodiment of the present invention further provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] In the method of the embodiment of the present application, voiceprint registration-free noise reduction means that in the process of real-time voice transmission, there is no need to register the speaker's voiceprint information in advance, so that only the audio of the speaker can be retained and the voices of other speakers can be filtered out. In the process of voice interaction, interfering voices and noise can be better suppressed by registering the speaker's voiceprint without feeling. During the call, the speaker's identity does not need to be registered, and the other end can still clearly hear the voice of the registered speaker without being disturbed by other people's voices. The audio of the registered speaker can be well retained and the interference of other speakers can be effectively suppressed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0011] Figure 1 A flowchart of a registration-free voiceprint noise reduction method provided by an embodiment of the present invention; Figure 2 A flowchart of another embodiment of a registration-free voiceprint noise reduction method provided by an embodiment of the present invention; Figure 3A flowchart of another embodiment of a registration-free voiceprint noise reduction method provided by an embodiment of the present invention; Figure 4 A flow chart of clustering of a registration-free voiceprint noise reduction method provided by an embodiment of the present invention; Figure 5 A schematic diagram showing a performance comparison between registration clustering and active clustering of a registration-free voiceprint noise reduction method provided by an embodiment of the present invention; Figure 6 A schematic diagram of a cluster registration test of a registration-free voiceprint noise reduction method provided by an embodiment of the present invention; Figure 7 A schematic diagram of a registration test captured by a registration-free voiceprint noise reduction method provided by an embodiment of the present invention; Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0014] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0015] In the present invention, "module", "device", "system" and the like refer to related entities applied to computers, such as hardware, a combination of hardware and software, software or software in execution, etc. In detail, for example, an element can be, but is not limited to, a process, a processor, an object, an executable element, an execution thread, a program and / or a computer running on a processor. In addition, an application program or a script program, a server running on a server can be an element. One or more elements can be in an execution process and / or thread, and an element can be localized on a computer and / or distributed between two or more computers, and can be operated by various computer-readable media. An element can also communicate through a local and / or remote process according to a signal with one or more data packets, for example, a signal from a data that interacts with another element in a local system, a distributed system, and / or a network on the Internet through a signal and interacts with other systems.

[0016] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include" and "comprise" include not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements.

[0017] The embodiment of the present invention provides a registration-free voiceprint noise reduction method, which can be applied to electronic devices. The electronic device can be a computer, a server, or other electronic products, etc., and the present invention is not limited to this.

[0018] Please refer to Figure 1 , which shows a registration-free voiceprint noise reduction method provided by an embodiment of the present invention.

[0019] like Figure 1 As shown, in step 101, valid audio that meets the total duration threshold during a real-time call is obtained, and the valid audio is clustered to obtain at least one type of sample points; In step 102, a sample point of the type with the most sample data among the at least one type of sample points is obtained, and valid audios of the sample points of the type with the most sample data are spliced ​​for voiceprint registration to obtain voiceprint information, wherein the valid audios of the sample points of the type with the most sample data are the audios of the speaker of the call; In step 103, the trained denoising neural network is assisted based on the voiceprint information, so that the denoising neural network learns the difference of the voiceprint features of each frame in the voiceprint information to obtain the denoised speaker audio.

[0020] In this embodiment, for step 101, valid audio that meets the total duration threshold is obtained during a real-time call, and the valid audio is clustered to obtain at least one type of sample points. Valid audio is not limited to being obtained in real-time calls, but can also be obtained in normal voice interactions. As long as the total duration threshold is met, it is valid audio. For example, during a call, voice activity detection (VAD) collects valid audio in real time to obtain valid audio that meets the total duration threshold during a real-time call. Clustering is performed when the effective audio duration meets the total duration threshold, and the clustering process includes frame processing, overlap factor detection, feature extraction, and voiceprint extraction. After clustering of the valid audio is completed, at least one type of sample points is obtained, and sample points of different types are different audio.

[0021] Afterwards, for step 102, a sample point of the type with the most sample data among at least one type of sample points is obtained, and the valid audio of the sample point of the type with the most sample data is spliced ​​for voiceprint registration to obtain voiceprint information, wherein the valid audio of the sample point of the type with the most sample data is the audio of the speaker of the call. For example, after clustering is completed to obtain sample points of multiple different types, the sample point of the type with the most sample data among all the sample points is used as the audio of the speaker. The audio sample points of the speaker are spliced ​​for voiceprint registration to obtain the voiceprint information of the speaker's audio.

[0022] Finally, for step 103, the trained denoising neural network is assisted based on the voiceprint information, so that the denoising neural network learns the difference of the voiceprint features of each frame in the voiceprint information, and obtains the denoised audio of the speaker. For example, after obtaining the voiceprint information of the speaker's audio, the voiceprint information is used to assist the training of the trained denoising neural network, and the denoising neural network learns the difference of the voiceprint features of each frame in the voiceprint information, so as to improve the denoising neural network. The neural network is assisted by the voiceprint information to learn the difference of the voiceprint features in each frame, thereby retaining the speaker's voice and removing interference, and finally obtaining the denoised audio of the speaker.

[0023] In the process of real-time voice transmission, the method of the embodiment of the present application can retain only the audio of the speaker and filter out the voices of other speakers without registering the voiceprint information of the speaker in advance. In the process of voice interaction, interfering voices and noise can be better suppressed by registering the voiceprint of the speaker without feeling. During the call, the other end can still clearly hear the voice of the registered speaker without registering the speaker's identity, without being disturbed by the voices of other people. The audio of the registered speaker can be well retained and the interference of other speakers can be effectively suppressed.

[0024] Please refer to Figure 2 , which shows another registration-free voiceprint noise reduction method provided by an embodiment of the present invention. Figure 1 A flowchart of the steps further defined in step 101 of "clustering the valid audio".

[0025] like Figure 2 As shown, in step 201, the valid audio is processed by framing, and the audio after framing is sent to an overlapping sound detection model for detection to confirm whether the audio after framing is overlapping audio; In step 202, if the valid audio after the frame processing is overlapping audio, the overlapping audio is removed; In step 203, if the valid audio after the frame processing is not overlapping audio, the valid audio after the frame processing is retained.

[0026] In this embodiment, for step 201, the valid audio during the real-time call is framed, and the framed audio is sent to the overlapping sound detection model for detection to confirm whether the framed audio is overlapping audio; for example, during the call, the valid audio collected in real time by the voice activity detection is framed, wherein the frame length is 1.5s and the frame shift is 0.75s. The registration-free voiceprint noise reduction method of the present application adopts a frame length of 1.5s and a frame shift of 0.75s, and does not limit the specific frame length and frame shift. After the framing is completed, the overlapping sound detection model is used to detect whether the framed audio has overlapping audio.

[0027] Afterwards, for step 202, if the valid audio after the frame processing is overlapping audio, the overlapping audio is removed, and if it is overlapping audio, it is discarded; finally, for step 203, if the valid audio after the frame processing is not overlapping audio, the valid audio after the frame processing is retained.

[0028] The method of the embodiment of the present application can effectively remove overlapping audio from the framed audio by performing frame segmentation and overlapping audio detection processing on the valid audio during a call.

[0029] In some optional embodiments, clustering the effective audio further includes extracting features from the effective audio retained after framing, inputting the effective audio features into a pre-trained speaker extraction network, and extracting the speaker's embedded information features, wherein the speakers' embedded information features are all different. For example, feature extraction is performed on the framed audio, the speech is converted into frequency domain features, and useful feature information is extracted. The extracted features are then input into a trained speaker extraction network to extract the speaker's embedded feature information. The embedded features are speaker-distinguishing. The embedded cosine similarity of the same speaker is high, and the embedded information features of different speakers are all different.

[0030] In some optional embodiments, the embedded information features of the same speaker are clustered by spectral clustering, and the embedded information features of the speakers close to the cluster center are retained. The retained embedded information features of the speakers are sample points, wherein the embedded information features of the speakers far from the cluster center are removed by means of a card threshold. The embedding fragments of the same speaker are clustered together by spectral clustering, and the outliers far from the cluster center are removed by means of a card threshold, and the embedded information features of the speakers close to the cluster center are retained.

[0031] In some optional embodiments, the denoising neural network includes an encoder, an enhance plug-and-play unit and a decoder, and the loss function used in the denoising neural network training includes the SISNR loss scale-invariant signal-to-noise ratio of the predicted audio and the clean audio. For example, the noisy audio is input into the neural network and the mask of the quiet speech is predicted through the encoder, enhance and decoder, and the loss function is the SISNR loss of the predicted audio and the clean audio.

[0032] Please refer to Figure 3 , which shows that a training method for a denoising neural network provided by an embodiment of the present invention is used for a registration-free voiceprint denoising method.

[0033] like Figure 3 As shown, in step 301, the preset data is converted into noisy audio, wherein the preset data includes registered audio, clean audio, noisy audio and interference audio; In step 302, the noisy audio is input into the denoising neural network, the denoised audio output by the denoising neural network is obtained, and the mask of the clean audio is predicted.

[0034] In this embodiment, for step 301, the prepared data, the registered audio, the clean audio, the noisy audio, and the interference audio are converted into noisy audio by adding different signal-to-noise ratios and signal-to-interference ratios.

[0035] For step 302, the noisy audio is input into the neural network and the mask of the quiet speech is predicted through the encoder, enhance, and decoder, and the denoised audio output by the denoising neural network is obtained. The denoised audio is clean audio.

[0036] The method of the embodiment of the present application utilizes voiceprint information to assist the denoising neural network in learning, so that the denoising neural network learns the difference in voiceprint features in each frame, thereby retaining the speaker's voice and removing interference.

[0037] In some optional embodiments, the data with different signal-to-noise ratios and / or signal-to-interference ratios in the preset data are added to obtain a noisy audio; feature extraction is performed on the noisy audio, and the time domain voice information of the noisy audio feature is converted into the frequency domain. The prepared data, the registered audio, the clean audio, the noisy audio, and the interference audio are converted into a noisy audio by adding different signal-to-noise ratios and signal-to-interference ratios, and the noisy audio is converted into the frequency domain by feature extraction. Here we use the Hanning window, frame length 512 frame shift 256, and the length of the FFT is 512.

[0038] In some optional embodiments, the features of the registered audio are extracted, and the features of the registered audio are input into the denoising neural network to obtain the feature results of the registered audio output by the denoising neural network; the denoised audio is spliced ​​with the feature results of the registered audio to obtain the denoised speaker audio. After the features of the registered audio are extracted, it is input into the GRU network and then averaged in the time dimension, and then subjected to the same operation as the noisy audio, and the output of the registered audio after the network is spliced ​​after the noisy audio is input into the network. Currently, the more popular personalized noise reduction is a combination of speech enhancement and voiceprint information. The neural network uses voiceprint information to learn the differences in voiceprint features in each frame, thereby retaining the speaker's voice and removing interference.

[0039] It should be noted that the registration-free voiceprint noise reduction method provided by this application has the advantages of registration-free personalized noise reduction: no need to register in advance during the call, and the user can actively save the speaker's voiceprint ID through the UI prompt. The defects of the existing technology are unavoidable defects in deep learning for speech enhancement, so it is also the case for voice separation of speakers. Combining the results of domestic and foreign research institutions, this application proposes a method of integrating voiceprint information in the encoding process, so as to make full use of low-dimensional voiceprint information.

[0040] In recent years, the technology of artificial intelligence neural networks has developed rapidly, and has shown a trend of being superior to traditional technologies in many fields and scenarios. Deep learning methods (such as neural network-based speech enhancement technology) have improved in these aspects and can provide higher accuracy and real-time performance. These methods use large amounts of data for training, can better adapt to different noise conditions, and can reduce latency and computing resource requirements through model compression and acceleration technology.

[0041] Traditional signal processing solutions are based on rigorous formula derivation to find the optimal solution in a statistical sense. Therefore, they have good and stable performance in scenarios that meet the algorithm assumptions. This is also the reason why most industries choose traditional signal processing solutions. However, non-registered personalized noise reduction using traditional signal processing faces the following challenges.

[0042] Computational complexity: Traditional unregistered voice extraction methods, such as algorithms based on spectral subtraction or statistical modeling, usually have high computational complexity. These methods may not meet the low latency requirements in real-time processing because they require a lot of calculations and model updates.

[0043] Poor noise adaptability: Traditional methods may not be able to handle various background noises and non-ideal environments well. Under complex noise conditions, traditional methods may not be able to accurately extract human voices, resulting in poor voice quality.

[0044] Difficulty in model updating: Traditional methods usually rely on fixed models or parameters, which makes them perform poorly in the face of changing background noise or environment. Real-time applications need to be able to dynamically adjust and optimize models to adapt to environmental changes.

[0045] Latency issue: Real-time applications require very low latency during processing, but traditional algorithms may take a certain amount of time to process the audio signal and perform vocal extraction, which may not be fast enough for real-time applications.

[0046] High computing resource requirements: Some traditional methods have high computing resource requirements during the training and inference stages, which may not be suitable for resource-constrained real-time application scenarios.

[0047] Please refer to Figure 4 , which shows a flow chart of clustering of a registration-free voiceprint noise reduction method of the present invention.

[0048] like Figure 4 As shown, during the call, vad collects valid audio in real time and performs clustering when the valid audio duration meets the total duration threshold.

[0049] Clustering includes the following processes: Frame processing: The audio is framed after VAD, with a frame length of 1.5s and a frame shift of 0.75s.

[0050] Overlapping detection: The frame-processed audio is fed into the overlapping audio detection model, and any overlapping audio is discarded.

[0051] Extract Fbank: Extract Fbank features from the framed audio, convert the speech into frequency domain features, and extract useful feature information.

[0052] Extract Embedding: Input the extracted Fbank features into the trained speaker extraction network to extract the speaker's embedding information. The embedding feature has speaker discrimination. The embedding cosine similarity of the same speaker is high.

[0053] Sc clustering: cluster the embedding segments of the same speaker together through spectral clustering.

[0054] Clustering threshold: outliers that are far from the cluster center are removed by using the card threshold.

[0055] Through the above steps, the class with more sample points is clustered as the speaker, and the audio of this class is spliced ​​for voiceprint registration.

[0056] The personalized noise reduction concept and principle provided by this application: Currently, the more popular personalized noise reduction is a combination of speech enhancement and voiceprint information. The neural network is assisted by voiceprint information to learn the differences in voiceprint features in each frame, thereby retaining the speaker's voice and removing interference.

[0057] step: 1) The prepared data, registered audio, clean audio, noisy audio, and interference audio are converted into noisy frequency by adding different signal-to-noise ratios and signal-to-interference ratios. The noisy frequency is converted into frequency domain by feature extraction. Here we use Hanning window, frame length 512, frame shift 256, and FFT length 512. 2) The noisy audio is input into the neural network and predicted through encoder, enhance, and decoder to produce a mask of quiet speech. The loss function is the SISNR loss of the predicted audio and the clean audio.

[0058] 3) After feature extraction, the registered audio is input into the GRU network and then averaged in the time dimension. It then undergoes the same operations as the noisy audio, and the output of the registered audio after the network is spliced ​​after the noisy audio is input into the network.

[0059] Advantages of personalized noise reduction without registration: No need to register in advance during a call, users can actively save the speaker's voiceprint ID through UI prompts.

[0060] Defects: The defects of the existing technology are unavoidable defects when deep learning is used for speech enhancement, and therefore the same is true for voice separation of speakers. Combining the results of domestic and foreign research institutions, this application proposes a method of integrating voiceprint information during the encoding process, so as to make full use of low-dimensional voiceprint information.

[0061] Please refer to Figure 5 , Figure 6 and Figure 7 , which shows a schematic diagram of the performance comparison between registration clustering and active clustering of a registration-free voiceprint noise reduction method of the present invention.

[0062] Figure 5 This is a piece of audio and the result after clustering. The test results are as follows: Figure 6 Cluster registration test, Figure 7 Captured 15-second registration test.

[0063] from Figure 6 and Figure 7 It can be seen from the spectrum in that the performance of active registration and clustering registration are almost the same.

[0064] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of combined actions, but those skilled in the art should be aware that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0065] In some embodiments, an embodiment of the present invention provides a non-volatile computer-readable storage medium, in which one or more programs including execution instructions are stored. The execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute any of the above-mentioned registration-free voiceprint noise reduction methods of the present invention.

[0066] In some embodiments, an embodiment of the present invention further provides a computer program product, comprising a computer program stored on a non-volatile computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer executes any one of the above-mentioned registration-free voiceprint noise reduction methods.

[0067] In some embodiments, an embodiment of the present invention further provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform a registration-free voiceprint noise reduction method.

[0068] Figure 8 is a schematic diagram of the hardware structure of an electronic device for executing a registration-free voiceprint noise reduction method provided by another embodiment of the present application, such as Figure 8 As shown, the device includes: One or more processors 810 and memory 820, Figure 8 A processor 810 is taken as an example.

[0069] The device for performing the registration-free voiceprint noise reduction method may further include: an input device 830 and an output device 840 .

[0070] The processor 810, the memory 820, the input device 830 and the output device 840 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0071] The memory 820, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the registration-free voiceprint noise reduction method in the embodiment of the present application. The processor 810 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 820, that is, the registration-free voiceprint noise reduction method in the above method embodiment is implemented.

[0072] The memory 820 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required by at least one function; the data storage area may store data created according to the use of the registration-free voiceprint noise reduction device, etc. In addition, the memory 820 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 820 may optionally include a memory remotely arranged relative to the processor 810, and these remote memories may be connected to the registration-free voiceprint noise reduction device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0073] The input device 830 can receive input digital or character information and generate signals related to user settings and function control of the registration-free voiceprint noise reduction device. The output device 840 can include display devices such as display screens.

[0074] The one or more modules are stored in the memory 820, and when executed by the one or more processors 810, the registration-free voiceprint noise reduction method in any of the above method embodiments is executed.

[0075] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0076] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0077] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0078] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0079] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A registration-free voiceprint call noise reduction method, comprising: Acquire valid audio that meets a total duration threshold during a real-time call, and cluster the valid audio to obtain at least one type of sample points; Obtain a type of sample point with the most sample data from the at least one type of sample points, and concatenate valid audios of the type of sample points with the most sample data for voiceprint registration to obtain voiceprint information, wherein the valid audios of the type of sample points with the most sample data are the audios of the speaker of the call; The trained denoising neural network is assisted based on the voiceprint information, so that the denoising neural network learns the difference of the voiceprint features of each frame in the voiceprint information to obtain the denoised speaker audio.

2. The method according to claim 1, wherein: The clustering of the valid audio comprises: Processing the valid audio by framing, and sending the audio after framing to an overlapping sound detection model for detection to confirm whether the audio after framing is overlapping audio; If the valid audio after the frame processing is overlapping audio, removing the overlapping audio; If the valid audio after the frame processing is not overlapping audio, the valid audio after the frame processing is retained.

3. The method according to claim 2, wherein: Clustering the valid audio further includes: Feature extraction is performed on the valid audio retained after the frame processing, and the valid audio features are input into a pre-trained speaker extraction network to extract the embedded information features of the speakers, wherein the embedded information features of the speakers are all different.

4. The method according to claim 3, wherein: After extracting features from the audio retained after the frame processing, the method further comprises: The embedded information features of the same speaker are clustered by spectral clustering, and the embedded information features of the speakers close to the cluster center are retained. The retained embedded information features of the speakers are sample points, wherein the embedded information features of the speakers far from the cluster center are removed by means of card threshold.

5. The method according to claim 1, wherein: The denoising neural network includes an encoder, a plug-and-play unit and a decoder, and the loss function used in training the denoising neural network includes a scale-invariant signal-to-noise ratio of predicted audio and clean audio.

6. The method according to claim 1, wherein: The training method of the denoising neural network includes: Converting preset data into noisy audio, wherein the preset data includes registered audio, clean audio, noisy audio and interference audio; The noisy audio is input into the denoising neural network, the denoised audio output by the denoising neural network is obtained, and the mask of the clean audio is predicted.

7. The method according to claim 6, wherein: The converting the preset data into a noisy frequency comprises: Adding data with different signal-to-noise ratios and / or signal-to-interference ratios in the preset data to obtain the noisy frequency; Feature extraction is performed on the noisy audio, and speech information in the time domain of the noisy audio feature is converted into frequency domain.

8. The method according to claim 1, wherein: The step of assisting the trained denoising neural network based on the voiceprint information so that the denoising neural network learns the difference in voiceprint features of each frame in the voiceprint information includes: Extracting features of the registered audio, inputting the features of the registered audio into the denoising neural network, and obtaining feature results of the registered audio output by the denoising neural network; The de-noised audio is concatenated with the feature result of the registered audio to obtain the de-noised speaker audio.

9. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in any one of claims 2 to 8.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 2 to 8 are implemented.

Images