A voiceprint noise reduction method, electronic device and storage medium

Abstract

Embodiments of the present application provide a voiceprint noise reduction method, an electronic device and a storage medium, relating to the technical field of audio processing. The method comprises: receiving a first operation; in response to the first operation, starting a voice call; performing signal enhancement processing on audio collected in the voice call process to obtain mixed audio; performing voiceprint self-learning on the mixed audio through a voiceprint self-learning model; if the duration of the voice call reaches a first preset duration and the target voiceprint template of a target speaker is not learned, performing NN noise reduction on the mixed audio to obtain a first NN noise reduction result; inputting the mixed audio and a preliminary voiceprint template into a voiceprint noise reduction model to obtain a first mask corresponding to the mixed audio, multiplying the frequency domain amplitude spectrum of the mixed audio with the first mask to obtain audio after voiceprint noise reduction; and fusing the audio after voiceprint noise reduction with the first NN noise reduction result and transmitting the first audio to a peer electronic device of the voice call. The voice call process can be improved in terms of noise reduction effect.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a voiceprint noise reduction method, electronic device, and storage medium. Background Technology

[0002] When making voice calls in noisy environments, ambient noise can severely impact voice quality. Current noise reduction methods can suppress common noises such as keyboard clicks and horns, but they are not very effective at suppressing human voice interference, especially clear non-target human voices. This results in significant noise residue during voice calls, affecting the user experience. Summary of the Invention

[0003] The purpose of this application is to provide a voiceprint noise reduction method, electronic device, and storage medium to reduce residual noise during voice transmission and improve user experience. The specific technical solution is as follows:

[0004] In a first aspect, embodiments of this application provide a voiceprint noise reduction method applied to an electronic device. The method includes: receiving a first operation from a user to initiate a voice call; in response to the first operation, initiating a voice call; performing signal enhancement processing on audio collected during the voice call to obtain mixed audio; performing voiceprint self-learning on the mixed audio using a voiceprint self-learning model, wherein the voiceprint self-learning model is a trained neural network model for learning voiceprint templates; if the duration of the voice call reaches a first preset duration and no target voiceprint template of the target speaker is learned, then further processing is performed on the mixed audio. The first NN denoising result is obtained by performing NN denoising. The mixed audio and the initial voiceprint template are input into the voiceprint denoising model to obtain the first mask corresponding to the mixed audio. The frequency domain amplitude spectrum of the mixed audio is multiplied by the first mask to obtain the voiceprint denoised audio. The voiceprint denoising model is a trained neural network model for voiceprint denoising, and the initial voiceprint template is the optimal voiceprint information of the target speaker currently learned by the voiceprint self-learning model. The voiceprint denoised audio is fused with the first NN denoising result to obtain the first audio. The first audio is transmitted to the electronic device at the other end of the voice call.

[0005] This method first performs signal enhancement processing on the audio during a user's voice call to obtain mixed audio, thereby enhancing the audio signal and reducing conventional noise during the signal enhancement process. Then, it learns the initial voiceprint template and the target voiceprint template of the target speaker through a voiceprint self-learning process. This eliminates the need for the user to actively register their voiceprint information beforehand, making the operation more convenient and ensuring that the learned voiceprint template is the target speaker's voiceprint template during the current call, thus improving accuracy. Since the accuracy of the initial voiceprint template gradually improves during the voiceprint self-learning process before the target voiceprint template is learned, this embodiment can perform voiceprint noise reduction on the mixed audio using the current initial voiceprint template and then perform NN noise reduction on the mixed audio before the voice call duration reaches a first preset duration and the target speaker's voiceprint template is learned. The two noise reduction results are then fused to obtain the first audio, which is transmitted to the other end's electronic device. By fusing the two noise reduction results, the noise reduction accuracy before learning the target voiceprint template can be improved. Furthermore, it can maximize the noise reduction effect of voiceprint denoising before learning the target voiceprint template, that is, it can block out other noise besides the target speaker's voice in advance. As time goes by, the initial voiceprint template becomes more and more accurate, and the voiceprint denoising effect gradually improves, which can reduce noise residue during voice calls, improve the noise reduction performance during voice calls, and improve the user's call experience.

[0006] In one possible implementation, after performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio, the method further includes: if the duration of the voice call does not reach a first preset duration, performing NN noise reduction on the mixed audio to obtain a second NN noise reduction result; fusing the mixed audio and the second NN noise reduction result to obtain a second audio; and transmitting the second audio to the electronic device at the other end of the voice call.

[0007] This process is the first stage of noise reduction during voice calls. In the first stage, a smooth transition from front-end enhancement to NN noise reduction is achieved, avoiding significant differences in noise reduction after adding voiceprint noise reduction, thus improving the user experience.

[0008] In one possible implementation, after performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio, the method further includes: if the target speaker's voiceprint template has been learned, inputting the mixed audio and the target voiceprint template into the voiceprint denoising model to obtain a second mask corresponding to the mixed audio; multiplying the frequency domain amplitude spectrum of the mixed audio with the second mask to obtain a third audio; and transmitting the third audio to the other end of the voice call electronic device.

[0009] This process is the third stage of noise reduction during voice calls. After learning the target voiceprint template, the target voiceprint template can be used to accurately remove noise other than the voice of the non-target speaker, thereby improving the noise reduction effect.

[0010] In one possible implementation, the step of performing voiceprint self-learning on the mixed audio using a voiceprint self-learning model includes: performing VAD processing on the mixed audio that meets the speech quality conditions to obtain multiple audio segments; each time an audio segment of a cumulative second preset duration is obtained, inputting the audio segment of the cumulative second preset duration into the voiceprint self-learning model to obtain the first voiceprint information output by the voiceprint self-learning model; clustering and fusing the first voiceprint information with the voiceprint information of each speaker in the voiceprint template library; when the number of clustering and fusion of the second voiceprint information with the most clustering and fusion times in the voiceprint template library reaches a preset number threshold, using the second voiceprint information as the target voiceprint template of the target speaker.

[0011] Compared to the method of users actively registering their own voiceprint information, this method achieves voiceprint learning without the user's awareness, which is more technological and convenient.

[0012] In one possible implementation, the voiceprint template library includes the voiceprint information of each speaker during the voice call and the number of clustering fusions corresponding to each voiceprint information. The initial voiceprint template is the voiceprint information with the highest number of current clustering fusions in the voiceprint template library. The step of clustering and fusing the voiceprint information with the voiceprint information of each speaker in the voiceprint template library includes: calculating the similarity between the first voiceprint information and the voiceprint information of each speaker in the voiceprint template library respectively; if the similarity between the first voiceprint information and the third voiceprint information in the voiceprint template library is greater than a preset similarity threshold, then the first voiceprint information and the third voiceprint information are clustered and fused, and the number of clustering fusions of the third voiceprint information is incremented by 1.

[0013] By judging similarity, the voiceprint template of the target speaker can be accurately selected from the voiceprint information of multiple speakers in a multi-user scenario. This allows the noise other than the target speaker's voice to be eliminated based on the voiceprint template, thus improving the noise reduction effect.

[0014] In one possible implementation, the frequency domain amplitude spectrum of the first audio is the sum obtained by weighted summing of the frequency domain amplitude spectrum of the audio after speaker noise reduction and the frequency domain amplitude spectrum of the first NN noise reduction result.

[0015] Among them, the weight of the frequency domain amplitude spectrum of the audio after voiceprint noise reduction is the dynamic smoothing factor, and the weight of the frequency domain amplitude spectrum of the first NN noise reduction result is the difference between 1 and the dynamic smoothing factor.

[0016] The dynamic smoothing factor is positively correlated with the current maximum cluster fusion count, which is the cluster fusion count of the voiceprint information with the highest current cluster fusion count in the voiceprint template library.

[0017] Since the dynamic smoothing factor is positively correlated with the current maximum clustering fusion count, the larger the current maximum clustering fusion count, the more accurate the learned initial voiceprint template, and the greater the influence of the frequency domain amplitude spectrum of the voiceprint-denoised audio on the first audio. Correspondingly, the smaller the influence of the frequency domain amplitude spectrum of the first NN denoising result on the first audio. This achieves a smooth transition from NN denoising to voiceprint denoising, avoiding the problem of significant differences in call quality before and after suddenly adding voiceprint denoising. Furthermore, it allows the denoising effect of voiceprint denoising to be demonstrated as early as possible.

[0018] In one possible implementation, the frequency domain amplitude spectrum of the first audio frequency... for:

[0019]

[0020] Wherein, S2 is the frequency domain amplitude spectrum of the audio after voiceprint noise reduction, S3 is the frequency domain amplitude spectrum of the first NN noise reduction result, and α is the dynamic smoothing factor.

[0021]

[0022] Where threshold is the preset number of times, c max This refers to the number of times the voiceprint information with the most current clustering and fusion times is clustered in the voiceprint template library.

[0023] As can be seen from the above formula, with c max As the threshold approaches, the value of α gradually increases and approaches 1. Correspondingly, the influence of the frequency domain amplitude spectrum S2 after voiceprint denoising on the first audio gradually increases, while the influence of the frequency domain amplitude spectrum S3 of the first NN denoising result on the first audio gradually decreases, achieving a smooth transition from NN denoising to voiceprint denoising. Furthermore, voiceprint denoising can begin before the target voiceprint template is fully learned. As the learned initial voiceprint template becomes more accurate, the voiceprint denoising effect also improves. This achieves gradual improvement of the voiceprint denoising effect according to the self-learning progress and maximizes the early manifestation of the denoising results, avoiding voiceprint denoising taking effect too late.

[0024] In one possible implementation, the frequency domain amplitude spectrum of the second audio is: the sum obtained by weighted summation of the frequency domain amplitude spectrum of the mixed audio and the frequency domain amplitude spectrum of the second NN noise reduction result;

[0025] Among them, the weight of the frequency domain amplitude spectrum of the mixed audio is negatively correlated with the duration of the voice call, while the weight of the frequency domain amplitude spectrum of the second NN noise reduction result is positively correlated with the duration of the voice call.

[0026] In other words, as the duration of the voice call increases, the influence of the frequency domain amplitude spectrum of the mixed audio on the second audio gradually decreases, while the influence of the frequency domain amplitude spectrum of the second NN noise reduction result on the second audio gradually increases. This achieves a gradual transition from the noise reduction effect of the front-end enhancement to the noise reduction effect of NN noise reduction, which can avoid the problem of large differences in call quality caused by the sudden addition of voiceprint noise reduction later.

[0027] In one possible implementation, the frequency domain amplitude spectrum of the second audio... for:

[0028]

[0029] Where t0 is the time point at which the voiceprint noise reduction switch in the voice call interface is turned on after the voice call is initiated, t i t1 is the current time; t1 is the time point after the first preset duration when the voiceprint noise reduction switch is turned on; S0 is the frequency domain amplitude spectrum of the mixed audio; and S1 is the frequency domain amplitude spectrum of the second NN noise reduction result.

[0030] It can be seen that, with t i As we approach t1, the value to the left of the plus sign in the above formula gradually decreases, meaning the impact of the enhanced mixed audio from the front end on the final noise reduction result gradually diminishes. Furthermore, as t... i As the value approaches t1, the value on the right side of the above formula gradually increases, meaning that the influence of the NN-denoised audio on the final denoising result gradually increases. This is equivalent to gradually transitioning from front-end enhanced denoising to NN denoising, resulting in a gradual transition in denoising effect and improved user experience.

[0031] In one possible implementation, the voiceprint self-learning model and the voiceprint noise reduction model are jointly trained based on the same training set, which includes mixed audio samples after signal enhancement processing.

[0032] This application employs a joint training approach to ensure consistency in the final feature mapping results of the two models. While training the voiceprint denoising model, the voiceprint self-learning model is still being optimized, allowing the embedding layer of the pre-trained voiceprint self-learning model to continue updating and becoming more compatible with the voiceprint denoising model. This results in higher consistency in the feature mapping results of the two models, thereby improving the final voiceprint denoising effect. Furthermore, both the voiceprint denoising model and the voiceprint self-learning model receive mixed audio after front-end enhancement processing as input. If they are trained separately, it is difficult for them to maintain consistency in feature mapping when the front-end enhancement processing algorithm changes. However, this application's joint training approach ensures that even if the front-end enhancement processing algorithm changes, the consistency in feature mapping between the voiceprint denoising model and the voiceprint self-learning model remains intact. Furthermore, the voiceprint denoising model and voiceprint self-learning model trained through joint training can be adapted to any front-end enhancement processing algorithm. The voiceprint denoising model and voiceprint self-learning model have stronger portability. Therefore, the embodiments of this application can decouple the voiceprint link from the call 3A link. That is, the voiceprint denoising model and voiceprint self-learning model included in the voiceprint link can be adapted to electronic devices with various different call 3A links, and can be flexibly deployed according to the memory and computing power of the electronic device.

[0033] In one possible implementation, after a voice call is initiated, the method further includes: displaying a voiceprint noise reduction switch on the call interface; if the voiceprint noise reduction switch is in the on state, performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio; if the voiceprint noise reduction switch is in the off state, and a user's activation operation is received, then in response to the activation operation, performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio.

[0034] By displaying voiceprint noise reduction in the voice call interface, users can flexibly choose whether to enable voiceprint noise reduction in each voice call, which is more flexible and better meets users' voice call needs.

[0035] In one possible implementation, the current on / off state of the voiceprint noise cancellation switch is the same as the on / off state during the last voice call on the electronic device. This way, if the user's needs remain unchanged, there's no need to adjust the on / off state of the voiceprint noise cancellation switch each time, making it more convenient for the user.

[0036] In one possible implementation, after performing voiceprint self-learning on the mixed audio using a voiceprint self-learning model to obtain the target speaker's voiceprint template, the method further includes: updating the target voiceprint template once every preset period; or, updating the target voiceprint template once during each voice call.

[0037] Because the voice of the same speaker can change over time, voiceprint drift may occur. Updating the target voiceprint template can improve its accuracy, thereby enhancing noise reduction. Updating the target voiceprint template at preset intervals balances computational resource consumption with accuracy. Updating the target voiceprint template during each voice call is suitable for scenarios with frequently changing voice call environments, maximizing the accuracy of the target voiceprint template used in each call and improving noise reduction.

[0038] In one possible implementation, the electronic device includes an ADSP module and an AP module. The ADSP module includes a front-end enhancement module, and the AP includes the voiceprint self-learning model and the voiceprint noise reduction model. The step of performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio includes: performing signal enhancement processing on the audio collected during the voice call through the front-end enhancement module to obtain the mixed audio.

[0039] The ADSP module includes a call 3A link, which is the front-end enhancement module used to enhance audio signals. The AP module includes a voiceprint link, which comprises the aforementioned voiceprint self-learning model and voiceprint noise reduction model.

[0040] Using this approach, both the voiceprint self-learning model and the voiceprint noise reduction model are decoupled from the front-end enhancement module, allowing for flexible deployment based on factors such as computing power and functional requirements. Furthermore, the effectiveness of the voiceprint self-learning model and the voiceprint noise reduction model does not depend on the front-end enhancement algorithm, making them highly adaptable.

[0041] Thirdly, embodiments of this application provide an electronic device, including: one or more processors and a memory; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the methods described in the first and second aspects.

[0042] Fourthly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are configured to invoke computer instructions to cause the electronic device to perform the methods described in the first and second aspects.

[0043] Fifthly, embodiments of this application provide a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform the methods described in the first and second aspects.

[0044] In a sixth aspect, embodiments of this application provide a computer program product, the computer program product comprising: computer program code, which, when executed on an electronic device, causes the electronic device to perform the methods described in the first and second aspects.

[0045] Understandably, the electronic device provided in the third aspect, the chip system provided in the fourth aspect, the computer storage medium provided in the fifth aspect, and the computer program product provided in the sixth aspect are all used to execute the method provided in this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0047] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0048] Figure 2 A schematic diagram of the software structure of an electronic device provided in this application embodiment.

[0049] Figure 3a A schematic diagram of a settings interface provided in an embodiment of this application;

[0050] Figure 3b A schematic diagram of a voice call interface provided in an embodiment of this application;

[0051] Figure 4 A flowchart of a voiceprint noise reduction method provided in an embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the structure of the voiceprint module provided in the embodiments of this application;

[0053] Figure 6 A schematic diagram illustrating the time relationship of the three noise reduction stages provided in the embodiments of this application;

[0054] Figure 7 A flowchart illustrating another voiceprint noise reduction method provided in this application embodiment;

[0055] Figure 8 A schematic diagram illustrating the implementation process of voiceprint self-learning and third-stage noise reduction provided in an embodiment of this application;

[0056] Figure 9 A flowchart illustrating the voiceprint self-learning process and noise reduction process provided in an embodiment of this application;

[0057] Figure 10 A schematic diagram of a process for obtaining a voiceprint template provided in an embodiment of this application;

[0058] Figure 11 A schematic diagram of a detailed functional interface provided for an embodiment of this application;

[0059] Figure 12 A schematic diagram illustrating a joint training method provided in an embodiment of this application;

[0060] Figure 13 This is an interactive flowchart of a voiceprint noise reduction method provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0062] The voiceprint noise reduction method provided in this application is applied to electronic devices, such as mobile phones, tablets, televisions, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other electronic devices that support voice calls.

[0063] To better understand the embodiments of this application, the structure of the electronic device of the embodiments of this application is described below.

[0064] like Figure 1 As shown, Figure 1 This is a schematic diagram of an electronic device provided in an embodiment of this application. Figure 1The electronic device shown may include a processor 110, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, sensor module 180, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, button 190, motor 191, indicator 192, camera 1-N 193, display screen 194, and subscriber identification module (SIM) card interface 1-N 195, etc.

[0065] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0066] Processor 110 may include one or more processing units, such as application processors, modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, memory, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0067] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0068] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0069] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include inter-integrated circuit (I2C) interfaces, inter-integrated circuit sound (I2S) interfaces, pulse code modulation (PCM) interfaces, universal asynchronous receiver / transmitter (UART) interfaces, mobile industry processor interfaces (MIPI), general-purpose input / output (GPIO) interfaces, etc.

[0070] The wireless communication function of electronic devices can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0071] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0072] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G in electronic devices. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0073] In the embodiments of the application, the mobile communication module may also be referred to as a cellular module, and the two can be described interchangeably.

[0074] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to a speaker, receiver, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0075] The wireless communication module 160 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (WiFi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2. In some embodiments, at least some functional modules of the wireless communication module 160 can be housed within processor 110.

[0076] In some embodiments of this application, an electronic device can establish a wireless connection with other electronic devices via a wireless communication module 160 (such as a Bluetooth module and a WLAN module) and an antenna 2 to enable data transmission between the electronic device and other electronic devices. For example, during a voice call, the electronic device can transmit audio data with other electronic devices via the wireless communication module 160.

[0077] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the electronic device to communicate with networks and other devices via wireless communication technology.

[0078] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0079] The charging management module 140 receives charging input from the charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0080] Electronic devices implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0081] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device may include one or N display screens 194, where N is a positive integer greater than 1.

[0082] Electronic devices can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0083] The ISP is used to process data fed back by the camera 193. The camera 193 is used to capture still images or videos. In some embodiments, the electronic device may include one or N cameras 193, where N is a positive integer greater than 1.

[0084] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device is selecting a frequency, a DSP can perform a Fourier transform on the frequency energy.

[0085] Electronic devices can implement audio functions through audio module 170, speakers, receivers, microphones, headphone jacks, and application processors. Examples include music playback, recording, and voice calls. For instance, during a voice call, the electronic device can collect audio data through the microphone. The collected audio data is then processed by the audio module 170 and the noise reduction function provided in this embodiment before being transmitted to the electronic device at the other end of the voice call. This allows the other electronic device to play the target speaker's audio in real time through a speaker or headphones.

[0086] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the electronic device.

[0087] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as screen mirroring, network sharing, etc.), etc. The data storage area may store data created during the use of the electronic device (such as video data, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0088] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The electronic device can receive button input and generate key signal inputs related to user settings and function control of the electronic device.

[0089] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, or to indicate messages, missed calls, notifications, etc.

[0090] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device. The electronic device can support one or N SIM card interfaces, where N is a positive integer greater than 1.

[0091] The software system of the aforementioned electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to illustrate the software system of the electronic device. Figure 2 As shown, the layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers.

[0092] From top to bottom, they are the application layer, application framework layer, Android Runtime and system libraries, and kernel layer.

[0093] The application layer can include a series of application packages. For example... Figure 2As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, Bluetooth, music, video, and SMS.

[0094] The application framework layer provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and algorithm framework, etc.

[0095] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0096] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0097] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build the display interface of an application. The display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0098] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0099] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and so on.

[0100] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0101] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0102] The core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part is the Android core library. The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0103] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0104] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0105] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0106] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0107] A 2D graphics engine is a drawing engine for 2D drawing.

[0108] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, and sensor drivers.

[0109] like Figure 2 As shown, the kernel layer is connected to the hardware layer. The hardware layer includes a system-on-chip (SOC), which includes an advanced digital signal processor (ADSP) module and an application processor (AP) module.

[0110] The ADSP module deploys a Call 3A link, an algorithmic link used for front-end audio enhancement processing. The Call 3A link includes an Acoustic Echo Cancel (AEC) module, an Automatic Gain Control (AGC) module, a Noise Reduction (NR) module, and an Equalizer (EQ) module.

[0111] The AP module includes a voiceprint link (also known as a voiceprint module), which is an algorithm link. Specifically, it includes a voiceprint self-learning module and a voiceprint noise reduction module. The voiceprint self-learning module includes a voiceprint self-learning module, and the voiceprint noise reduction module includes a voiceprint noise reduction model.

[0112] Traditional noise reduction modules are typically deployed within the call 3A link of an ADSP module. Decoupling the noise reduction module from the call 3A link is difficult, and the computational power of the noise reduction module is constrained by the computational power of the ADSP module, affecting the noise reduction effect. In this embodiment, both the voiceprint self-learning module and the voiceprint noise reduction module are decoupled from the call 3A link. This allows them to operate without being constrained by the computational power of the ADSP module. Furthermore, the inputs to both the voiceprint self-learning model and the voiceprint noise reduction model are mixed audio processed by the call 3A link front-end enhancement. By jointly training the voiceprint self-learning model and the voiceprint noise reduction model, the consistency of their feature mapping is maximized. Therefore, even with decoupling, the noise reduction effect is guaranteed, overcoming the difficulty of decoupling.

[0113] The voiceprint noise reduction method provided in this application is applied to voice calls between electronic devices, such as voice calls using social software or telephone calls using a SIM card. During a voice call, the electronic device used by the target speaker can collect the current audio and perform noise reduction processing on the collected audio, thereby transmitting the noise-reduced audio to the other electronic device, so that the other user can hear the target speaker's voice clearly.

[0114] When the electronic device is not connected to headphones, its microphone can capture the current audio. When headphones are connected, the headphones' microphone can capture the current audio and provide it to the electronic device. The headphones connected to the electronic device can be wired or wireless; wireless headphones can be Bluetooth headphones, bone conduction headphones, etc.

[0115] The audio received by electronic devices may include ambient noise and human voice noise.

[0116] As an example, if the target speaker is making a voice call in an outdoor public place, the environmental noise may include car horns, music and broadcasts from roadside shops, and human voice noise may include the voices of passersby and companions.

[0117] For example, if the target speaker is having a conference with others in a meeting room via audio conferencing, the ambient noise may include keyboard typing, and the human voice noise may include the voices of other people in the meeting room.

[0118] For example, if the target speaker is having a voice call with someone at home, the ambient noise may include the sound of the television, and the human voice noise may include the sound of family members talking.

[0119] In this embodiment, the user can choose whether to enable the noise reduction function according to their own needs. If the user needs to enable the noise reduction function, then... Figure 3a As shown, users can open the phone's settings interface and turn on the call noise reduction mode switch 301.

[0120] With call noise reduction mode switch 301 turned on, subsequently, at the start of a voice call, such as Figure 3b As shown, the electronic device can display the voiceprint noise reduction switch 302 on the voice call interface. If it receives a user's activation operation on the voiceprint noise reduction switch 302, it will respond to the activation operation by performing noise reduction processing on the voice call according to the voiceprint noise reduction method provided in this application embodiment. If it does not receive a user's activation operation on the voiceprint noise reduction switch 302, that is, the voiceprint noise reduction switch 302 is in the off state, it will use the NR module in the call 3A link to perform noise reduction processing on the voice call.

[0121] It should be noted that if a user turns on the voiceprint noise reduction switch 302 during a voice call, the voiceprint noise reduction switch 302 will be on by default during the next voice call. Users can choose not to operate the noise reduction switch 302 according to their actual needs, that is, to keep the voiceprint noise reduction switch 302 on, or they can choose to turn off the voiceprint noise reduction switch 302.

[0122] Optionally, the voiceprint noise reduction switch 302 can automatically disappear after a specified duration, such as 3 seconds. After the voiceprint noise reduction switch 302 disappears, during a voice call, if the user needs to turn the voiceprint noise reduction switch 302 on or off, they can swipe down from the top of the screen to make the electronic device display the voiceprint noise reduction switch 302 again.

[0123] like Figure 4 As shown, the voiceprint noise reduction method provided in this application embodiment specifically includes the following steps:

[0124] S401, Receive the user's first operation to initiate a voice call.

[0125] The first operation is either the user answering a voice call or the user successfully making a voice call. A user successfully making a voice call means that the user initiates a voice call to the other end of the electronic device, and the other end of the electronic device answers the call.

[0126] S402, In response to the first operation, initiate a voice call.

[0127] S403. Perform signal enhancement processing on the audio collected during the voice call to obtain mixed audio.

[0128] Signal enhancement processing refers to using the call 3A link in the ADSP module to process the audio captured by the microphone, thereby enhancing the audio signal. Signal enhancement processing can include AEC, AGC, NR, and EQ. After signal enhancement processing, the resulting mixed audio includes the target speaker's audio as well as noise audio, such as human voice noise.

[0129] S404. Perform voiceprint self-learning on the mixed audio using a voiceprint self-learning model.

[0130] Among them, the voiceprint self-learning model is a trained neural network model used to learn voiceprint templates.

[0131] The voiceprint self-learning model in this embodiment requires more computing power than the signal enhancement processing. Therefore, the voiceprint self-learning model can be deployed in the AP of an electronic device to support greater computing power.

[0132] During the voiceprint self-learning process, the voiceprint information of the target speaker can be obtained, and as the voiceprint self-learning progresses, the learned voiceprint information will become more and more accurate.

[0133] In this embodiment, the finally learned voiceprint information that meets the accuracy criteria is used as the target voiceprint template for the target speaker. Furthermore, before obtaining the target voiceprint template, the currently learned optimal voiceprint information for the target speaker is used as the initial voiceprint template. The method for evaluating the accuracy of the voiceprint information will be described in detail in subsequent embodiments.

[0134] S405. If the duration of the voice call reaches the first preset duration and the target speaker's voiceprint template has not been learned, then the mixed audio is denoised using a neural network (NN) to obtain the first NN denoising result.

[0135] S406. If the duration of the voice call reaches the first preset duration and the target speaker's voiceprint template has not been learned, then the mixed audio and the initial voiceprint template are input into the voiceprint denoising model to obtain the first mask corresponding to the mixed audio. The amplitude spectrum of the mixed audio is multiplied by the first mask to obtain the audio after voiceprint denoising.

[0136] The voiceprint denoising model is a trained neural network model for voiceprint denoising. The first mask is used to extract the target speaker's audio from the mixed audio. Multiplying the amplitude spectrum of the mixed audio with the first mask allows the extraction of the target speaker's audio, i.e., the denoised audio, from the mixed audio. It is understood that the denoised audio does not include any noise other than the target speaker's voice.

[0137] The voiceprint noise reduction model of this application embodiment is also deployed in the AP of the electronic device to support greater computing power.

[0138] It should be noted that S405-S406 can be executed in parallel or in a sequential order, and this application embodiment does not limit this. In one implementation, when it is determined that the duration of the voice call needs to reach a first preset duration and no voiceprint learning template of the target speaker has been learned, NN noise reduction and voiceprint noise reduction of the mixed audio can be performed in parallel to improve noise reduction efficiency.

[0139] S407. The audio after voiceprint noise reduction is fused with the first NN noise reduction result to obtain the first audio.

[0140] S408: Transmit first audio to the electronic device at the other end of the voice call.

[0141] This method first performs signal enhancement processing on the audio during a user's voice call to obtain mixed audio, thereby enhancing the audio signal and reducing conventional noise during the signal enhancement process. Then, it learns the initial voiceprint template and the target voiceprint template of the target speaker through a voiceprint self-learning process. This eliminates the need for the user to actively register their voiceprint information beforehand, making the operation more convenient and ensuring that the learned voiceprint template is the target speaker's voiceprint template during the current call, thus improving accuracy. Since the accuracy of the initial voiceprint template gradually improves during the voiceprint self-learning process before the target voiceprint template is learned, this embodiment can perform voiceprint noise reduction on the mixed audio using the current initial voiceprint template and then perform NN noise reduction on the mixed audio before the voice call duration reaches a first preset duration and the target speaker's voiceprint template is learned. The two noise reduction results are then fused to obtain the first audio, which is transmitted to the other end's electronic device. By fusing the two noise reduction results, the noise reduction accuracy before learning the target voiceprint template can be improved. Furthermore, it can maximize the noise reduction effect of voiceprint denoising before learning the target voiceprint template, that is, it can block out other noise besides the target speaker's voice in advance. As time goes by, the initial voiceprint template becomes more and more accurate, and the voiceprint denoising effect gradually improves, which can reduce noise residue during voice calls, improve the noise reduction performance during voice calls, and improve the user's call experience.

[0142] Understandably, the time required to learn the target voiceprint template depends on factors such as the call environment and the presence of human voice interference. Before learning the target voiceprint template, it is impossible to use the target voiceprint template for voiceprint noise reduction.

[0143] For example, when a user starts a call, the learning of the target voiceprint template begins, and this learning is completed in the 5th minute of the call. In this scenario, voiceprint noise reduction cannot be performed using the target voiceprint template in the first 5 minutes of the call; that is, human voice interference still exists in the first 5 minutes of the call audio, and voiceprint noise reduction takes effect relatively late. In a conference room scenario, the target voiceprint template needs to be learned for each voice call. Consequently, voiceprint noise reduction cannot be performed before the target voiceprint template is learned, resulting in a relatively late take-off time for voiceprint noise reduction during each meeting.

[0144] Furthermore, if only the audio is enhanced in the first 5 minutes of the call, and then voiceprint noise reduction is suddenly added in the 6th minute, the audio quality will be significantly different between the first 5 minutes and the last 5 minutes, which will affect the user's voice call experience.

[0145] To avoid the problems of delayed voiceprint noise reduction and significant differences in audio quality before and after noise reduction, embodiments of this application can utilize a voiceprint module to perform noise reduction through a multi-stage noise reduction method. For example... Figure 5 As shown, the voiceprint module in this embodiment includes a voiceprint self-learning module, a voiceprint noise reduction module, and a neural network noise reduction module. The voiceprint self-learning module learns the target voiceprint template, the voiceprint noise reduction module performs voiceprint noise reduction using either the initial voiceprint noise reduction template or the target voiceprint template, and the neural network noise reduction module performs supplementary noise reduction during the voiceprint learning phase. The audio processed by both the voiceprint noise reduction module and the neural network noise reduction module is mixed audio after front-end enhancement processing (signal enhancement processing).

[0146] In the first stage, front-end enhancement and neural network noise reduction are used for fused noise reduction.

[0147] In the second stage, NN noise reduction and the initial voiceprint template are used for fusion noise reduction, which is the noise reduction process in the above embodiment when the duration of the voice call reaches the first preset duration and the target voiceprint template has not been learned.

[0148] Based on the example above, if the learning of the target voiceprint template is completed in the 5th minute of the voice call, then the second stage belongs to the first 5 minutes of the voice call. In the second stage, it is possible to use NN noise reduction and the initial voiceprint template for fusion noise reduction, which can provide the maximum voiceprint noise reduction capability in the first 5 minutes of the voice call, and promptly reflect the results of voiceprint noise reduction, thereby improving the user experience.

[0149] In the third stage, target voiceprint templates are used for voiceprint noise reduction.

[0150] The time relationship of the above three stages is as follows: Figure 6 As shown.

[0151] The first stage begins at time t0 and ends at time t1. During this first stage, the voiceprint template has not yet been learned.

[0152] t0 is the moment when the voiceprint noise reduction switch in the voice call interface is turned on after the voice call is started. t1 is the time point after t0, which is the first preset duration. For example, if the first preset duration is two minutes, then t1 is the time two minutes after the moment when voiceprint noise reduction is turned on.

[0153] The second phase begins at time t1 and ends at time t2. In this phase, the initial voiceprint template has been learned, but the target voiceprint template has not yet been learned. t2 represents the moment the target voiceprint template is learned.

[0154] The third stage begins at time t2 and ends at time t3. In this third stage, the target voiceprint template has been learned. t3 marks the end of the voice call.

[0155] like Figure 7 As shown, the multi-stage noise reduction method provided in this application embodiment specifically includes the following steps:

[0156] S701, receives the user's first operation to initiate a voice call.

[0157] S702, responding to the first operation, initiates a voice call.

[0158] S703: Display the voiceprint noise reduction switch in the voice call interface.

[0159] It should be noted that if the voiceprint noise reduction switch is currently turned on, the electronic device can directly execute the step in S705 to perform signal enhancement processing on the audio collected during the voice call.

[0160] S704: Receives user's operation to turn on the voiceprint noise reduction switch.

[0161] It should be noted that if the voiceprint noise reduction switch is in the off state, and no user operation to turn it on is received within a specified time after the switch is displayed, the NR module in the call 3A link will be used to perform noise reduction processing on the audio collected during the voice call, or other methods in related technologies will be used for noise reduction processing, and the noise-reduced audio will be transmitted to the other end electronic device. This specified time can be the duration the voiceprint noise reduction switch is displayed, for example, 3 seconds.

[0162] If the voiceprint noise reduction switch is in the off state, and within a specified time after the voiceprint noise reduction switch is displayed, the user's operation to turn on the voiceprint noise reduction switch is received, then S705 continues to be executed.

[0163] S705, in response to the enabling operation, performs signal enhancement processing on the audio collected during the voice call to obtain mixed audio.

[0164] S706. Voiceprint self-learning is performed on mixed audio through a voiceprint self-learning model.

[0165] S707. Determine whether the duration of the voice call has reached the first preset duration.

[0166] Understandably, voice calls are continuous, and S707 can be executed each time a mixed audio signal is received during the call. Furthermore, there is no strict order between S707 and S706; execution of S706 stops once the target voiceprint template has been learned.

[0167] If the judgment result is negative, that is, the duration of the voice call has not reached the first preset duration, then it is determined that the current moment belongs to the first stage of noise reduction, and then S708-S710 are executed; if the judgment result is positive, that is, the duration of the voice call has reached the first preset duration, then S711 is executed.

[0168] S708. Perform NN noise reduction on the mixed audio to obtain the second NN noise reduction result.

[0169] In the first stage, the voiceprint self-learning module is performing voiceprint self-learning, but it has not yet learned the initial voiceprint template and the target voiceprint template. Therefore, the voiceprint noise reduction module is turned off and voiceprint noise reduction is temporarily suspended.

[0170] S709. The mixed audio and the second NN noise reduction result are fused to obtain the second audio.

[0171] In this step, the fusion method used is to fuse the frequency domain amplitude spectrum of the mixed audio with the frequency domain amplitude spectrum of the second NN noise reduction result. The frequency domain amplitude spectrum of the second audio is the sum obtained by weighted summation of the frequency domain amplitude spectrum of the mixed audio and the frequency domain amplitude spectrum of the second NN noise reduction result.

[0172] Among them, the weight of the frequency domain amplitude spectrum of the mixed audio is negatively correlated with the duration of the voice call, while the weight of the frequency domain amplitude spectrum of the second NN noise reduction result is positively correlated with the duration of the voice call.

[0173] The frequency domain amplitude spectrum of the second audio frequency Specifically:

[0174]

[0175] Where t0 is the time point when the voiceprint noise reduction switch in the voice call interface is turned on after the voice call is initiated. t1 is the time point after the first preset duration when the voiceprint noise reduction switch is turned on, i.e., t1 is t0 plus the first preset duration. i S0 represents the frequency domain amplitude spectrum of the mixed audio, and S1 represents the frequency domain amplitude spectrum of the second NN noise reduction result.

[0176] It can be seen that, with t i As we approach t1, the value to the left of the plus sign in the above formula gradually decreases, meaning the impact of the enhanced mixed audio from the front end on the final noise reduction result gradually diminishes. Furthermore, as t... i As the value approaches t1, the value on the right side of the above formula gradually increases, meaning that the influence of the NN-denoised audio on the final denoising result gradually increases. This is equivalent to gradually transitioning from front-end enhanced denoising to NN denoising, resulting in a gradual transition in denoising effect and improved user experience.

[0177] S710 transmits a second audio signal to the other end of a voice call.

[0178] S711. Determine whether the target speaker's target voiceprint template has been learned.

[0179] If not, determine that the current moment belongs to the second stage of noise reduction, and then execute S712-S715; if yes, determine that the current moment belongs to the third stage of noise reduction, and then execute S716-S718.

[0180] S712. Perform NN noise reduction on the mixed audio to obtain the first NN noise reduction result.

[0181] S713. Input the mixed audio and the initial voiceprint template into the voiceprint denoising model to obtain the first mask corresponding to the mixed audio. Multiply the amplitude spectrum of the mixed audio with the first mask to obtain the audio after voiceprint denoising.

[0182] It should be noted that in the second stage, an initial version of the voiceprint template has been obtained. The initial version of the voiceprint template is the optimal voiceprint information of the target speaker that has been learned so far. However, the initial version of the voiceprint template cannot accurately represent the voiceprint information of the target speaker. That is, the final target voiceprint template has not yet been learned. At this time, the voiceprint self-learning module can continue to iterate and learn.

[0183] Because the voiceprint self-learning module is still iterating, the initial voiceprint templates obtained at different points in time during the second stage may be different.

[0184] S714. The audio after voiceprint noise reduction is fused with the first NN noise reduction result to obtain the first audio.

[0185] In this step, the fusion method used is to fuse the frequency domain amplitude spectrum of the audio after voiceprint noise reduction with the frequency domain amplitude spectrum of the first NN noise reduction result. That is, the frequency domain amplitude spectrum of the first audio is the sum obtained by weighted summation of the frequency domain amplitude spectrum of the audio after voiceprint noise reduction and the frequency domain amplitude spectrum of the first NN noise reduction result.

[0186] Among them, the weight of the frequency domain amplitude spectrum of the audio after voiceprint noise reduction is the dynamic smoothing factor, and the weight of the frequency domain amplitude spectrum of the first NN noise reduction result is the difference between 1 and the dynamic smoothing factor.

[0187] The dynamic smoothing factor is positively correlated with the current maximum cluster fusion count, which is the cluster fusion count of the voiceprint information with the highest current cluster fusion count in the voiceprint template library.

[0188] The frequency domain amplitude spectrum of the first audio audio. Specifically:

[0189]

[0190] Where S2 is the frequency domain amplitude spectrum of the audio after voiceprint noise reduction, S3 is the frequency domain amplitude spectrum of the first NN noise reduction result, and α is the dynamic smoothing factor.

[0191]

[0192] Where threshold is the preset number of times, c max This represents the number of times the voiceprint information with the most current clustering and fusion counts is used in the voiceprint template library.

[0193] During the voiceprint self-learning process, the voiceprint self-learning model outputs the recognized voiceprint information, which is stored in a voiceprint template library. This library specifically includes the voiceprint information of each speaker during the voice call and the number of clustering fusions corresponding to each voiceprint. Clustering fusion refers to the process where, if the similarity between the newly recognized voiceprint information and the voiceprint information already stored in the voiceprint template library exceeds a preset similarity threshold, these two voiceprints are clustered and fused, and the clustering fusion count of the stored voiceprint information is incremented by 1. In this embodiment, the initial voiceprint template is the voiceprint information with the highest current clustering fusion count in the voiceprint template library.

[0194] S715 transmits first audio to the electronic device at the other end of the voice call.

[0195] As can be seen from the above formula, with c max As the threshold approaches, the value of α gradually increases and approaches 1. Correspondingly, the influence of the frequency domain amplitude spectrum S2 after voiceprint denoising on the first audio gradually increases, while the influence of the frequency domain amplitude spectrum S3 of the first NN denoising result on the first audio gradually decreases, thus achieving a smooth transition from NN denoising to voiceprint denoising.

[0196] For example, starting from the 3rd minute of a voice call, the mixed audio can be denoised by NN and voiceprint denoising respectively. The audio transmitted to the other end electronic device is the audio after the NN denoising result and the voiceprint denoising result are fused. As the voiceprint self-learning module progresses, the human voice interference in the denoised audio becomes smaller and smaller. Before learning the target voiceprint template, it is already able to suppress human voice interference to a certain extent.

[0197] As can be seen, this method allows voiceprint noise reduction to begin before the target voiceprint template is fully learned. As the initial voiceprint template becomes more and more accurate, the voiceprint noise reduction effect will also improve. This achieves gradual improvement of the voiceprint noise reduction effect according to the self-learning progress and can maximize the early manifestation of the voiceprint noise reduction results, avoiding the voiceprint noise reduction taking effect too late.

[0198] S716. Input the mixed audio and the target voiceprint template into the voiceprint noise reduction model to obtain the second mask corresponding to the mixed audio.

[0199] In the third stage, an accurate target voiceprint template has been learned, eliminating the need for neural network noise reduction.

[0200] S717. Multiply the frequency domain amplitude spectrum of the mixed audio with the second mask to obtain the third audio.

[0201] S718, transmits third audio to the electronic device at the other end of the voice call.

[0202] Understandably, the third stage ends and the process concludes after the voice call ends.

[0203] It should be noted that in the above three stages, the noise reduction method in the front-end enhancement process is applicable to conventional steady-state noise, NN noise reduction can further process non-human voice interference on this basis, and voiceprint noise reduction can further suppress human voice interference, that is, the noise reduction effect is gradually progressive.

[0204] In the above process, a smooth transition from front-end enhanced noise reduction to neural network (NN) noise reduction was achieved in the first stage, and a smooth transition from NN noise reduction to voiceprint noise reduction in the second stage. In the third stage, the learned target noise reduction template can be used for voiceprint noise reduction. Thus, noise reduction of mixed audio can be better performed before the target voiceprint template is learned, allowing the noise reduction effect of voiceprint noise reduction to be demonstrated in advance. This also avoids the problem of significant differences in noise reduction when switching directly from front-end enhanced noise reduction to voiceprint noise reduction. The smooth transition between the three noise reduction methods improves the noise reduction effect and enhances the user experience.

[0205] In the embodiments of this application, the above-mentioned voiceprint self-learning and third-stage noise reduction implementation process is as follows: Figure 8 As shown, after a voice call begins, if the user's voiceprint noise reduction switch is detected, the acquired audio is first enhanced at the front end to obtain mixed audio. This mixed audio is then used as registration data to begin the registration process. During registration, the mixed audio undergoes data filtering to input only the audio that meets the speech quality requirements into the voiceprint self-learning model. The speaker's voiceprint information output by the voiceprint self-learning model is then stored in the voiceprint template library. Finally, the target speaker's target voiceprint template is retrieved from the voiceprint template library.

[0206] Furthermore, in the third stage of the noise reduction process, the target voiceprint template can be obtained. Then, feature extraction is performed on the mixed audio and the target speaker's voiceprint template. Specifically, feature extraction can be performed using Short-Time Discrete Cosine Transform (STDCT). After feature extraction, the extracted features are input into the voiceprint noise reduction model to obtain the second mask output by the model. Then, the frequency domain amplitude spectrum of the extracted features is multiplied by the second mask to obtain the noise-reduced audio features. Finally, the noise-reduced audio features are processed using Inverse Short-Time Discrete Cosine Transform (ISTDCT) to obtain the voiceprint-denoised audio.

[0207] The following provides a detailed description of the voiceprint self-learning process in the embodiments of this application.

[0208] like Figure 9 As shown, the original audio collected during the voice call is first enhanced by front-end processing (audio enhancement processing) to obtain mixed audio. Then, the mixed audio is denoised by NN. Finally, the audio after NN denoising is judged to determine whether the audio meets the audio quality conditions.

[0209] In this embodiment, it is possible to determine whether the audio meets the audio quality conditions based on the speech duration and signal-to-noise ratio. In actual implementation, the audio quality conditions may also include other conditions besides speech duration and signal-to-noise ratio. This embodiment does not limit this.

[0210] Specifically, for each audio segment, if the duration of the audio segment is less than the first duration (e.g., 0.5 seconds), then the audio segment is determined to not meet the speech quality conditions.

[0211] If the duration of the audio segment is greater than or equal to a first duration, then for each audio segment of the first duration included in the audio segment, it is determined whether the average signal-to-noise ratio of the audio segment is less than a preset signal-to-noise ratio threshold. If not, it is determined that the audio segment does not meet the speech quality conditions; if so, voice activity detection (VAD) processing is performed on the audio segment, and the VAD-processed audio segment is stored in the corpus. Audio segments that do not meet the speech quality conditions can be directly used as the audio output after NN noise reduction, and then transmitted to the other end electronic device.

[0212] Subsequently, it can be determined in real time whether the duration of audio segments in the corpus meets the requirements, that is, whether the cumulative duration of audio segments in the corpus reaches the second duration (cumulative second preset duration, for example, 2 seconds). If the requirements are met, the audio segments with the cumulative second duration are taken out of the corpus and input into the voiceprint self-learning model. The voiceprint information (first voiceprint information) output by the voiceprint self-learning model for the audio segments with the cumulative second duration is obtained. After multi-speaker clustering and fusion of the voiceprint information with the voiceprint information in the voiceprint template library, it is stored in the voiceprint template library.

[0213] Then, when there is voiceprint information in the voiceprint template library that has reached a preset threshold number of clustering and fusion times, the voiceprint information is used as the target voiceprint template of the target speaker, and the target voiceprint template is used to perform voiceprint noise reduction on the mixed audio to obtain the audio after voiceprint noise reduction.

[0214] The process of performing multi-speaker clustering and fusion on the voiceprint information in the voiceprint template library to obtain the final voiceprint template is as follows: Figure 10 As shown. First, the audio segment of the second duration is input into the voiceprint self-learning model to obtain the voiceprint information emb. new Then calculate emb respectively new Similarity to the voiceprint information of each speaker in the voiceprint template library.

[0215] The voiceprint template library includes voiceprint information from multiple speakers. i And each voiceprint information e i The corresponding number of cluster fusions, c i Here, i represents the number of the second-duration audio segment. It's understandable that in multi-user scenarios, audio from multiple speakers can be collected during a voice call, resulting in a voiceprint template library containing voiceprint information from multiple speakers. The existing voiceprint information in the template library includes voiceprints already identified during the current voice call, and may also include voiceprints identified in previous calls. For example, if a user enables voiceprint noise reduction, and the first voice call is short enough that the target speaker's voiceprint information hasn't been learned yet, the voiceprint information in the template library is retained. When the second voice call begins, the existing voiceprint information in the template library, along with newly added voiceprint information, is used to determine the target speaker's voiceprint information.

[0216] In this embodiment of the application, emb can be calculated. new Calculate the cosine similarity with the voiceprint information of each speaker, and update the index value i based on the cosine similarity.

[0217] i = argmaxcos(emb new e i ) = argmax(embnew ·e i / ||emb new ||*||e i ||)

[0218] Then emb new The voiceprint information e corresponding to the calculated subscript value i i Perform clustering and fusion, and combine the voiceprint information e i The corresponding cluster fusion count is increased by 1, i.e., c i =c i +1. The voiceprint information after clustering and fusion is as follows:

[0219] (e i )^=F fusion (e i emb new )

[0220] Understandably, if emb new If the cosine similarity with the voiceprint information of each speaker is less than a preset similarity threshold, then the emb will be... new The voiceprint information of a new speaker is added to the voiceprint template library, and the number of cluster fusions corresponding to this voiceprint information is set to 1.

[0221] Then, determine the maximum number of cluster fusions currently in the voiceprint template library, c. max =max(c i ), i = 1...n.

[0222] If the maximum number of cluster fusions is greater than the preset threshold, then the voiceprint information corresponding to the maximum number of cluster fusions will be used as the target voiceprint template of the target speaker.

[0223] The voiceprint self-learning method adopted in this application, compared to the method of users actively registering their own voiceprint information, achieves voiceprint learning without the user's awareness, which is more technologically advanced and convenient. Moreover, by judging similarity, it can accurately filter out the voiceprint template of the target speaker from the voiceprint information of multiple speakers in multi-user scenarios, so that subsequent noise other than the target speaker's voice can be eliminated based on the voiceprint template, thereby improving the noise reduction effect.

[0224] Furthermore, since the voice of the same speaker can change over time, there may be a problem of voiceprint drift. To overcome this problem, after learning the voiceprint template of the target speaker, the embodiments of this application can update the target voiceprint template at preset intervals during the user's use of the voiceprint noise reduction function, thereby achieving a better effect with each use.

[0225] For example, the preset period can be one month or three months, which can be set according to actual needs.

[0226] Before reaching the preset period, when a user makes a voice call, there is no need for voiceprint self-learning; voiceprint noise reduction can be performed directly using the target voiceprint template. After reaching the preset period, when the user makes a voice call, the voiceprint self-learning method described in the above embodiments can be used to relearn the target voiceprint template. In relearning the target voiceprint template, the voiceprint template library still retains the voiceprint information from the last voiceprint learning. New voiceprint information is added to the current voice call, thereby using this voiceprint information to learn a new target voiceprint template. This is equivalent to updating the target voiceprint template, making the target voiceprint template closer to the user's latest voiceprint, which is beneficial to improving the accuracy of subsequent voiceprint noise reduction.

[0227] In another implementation, for scenarios where the user's voice call environment changes frequently—for example, when the user frequently needs to make voice calls in different meetings, the participants in each meeting may change, and the surrounding noise level may vary frequently; or, in the same meeting room, the person using the voice call may be different each time—for example, employee A used the meeting room's shared account for a voice call on Monday morning, and employee B used the same account for a voice call on Monday afternoon, resulting in different target speakers in these two voice calls—users can also choose to update the target voiceprint template during each voice call as needed. That is, the target voiceprint template is learned once at the start of each voice call, thereby ensuring the accuracy of the voiceprint noise reduction results during each voice call.

[0228] Users can select the update method for the target voiceprint template through the detailed function interface of the call noise reduction mode in the settings interface. For example, users can click Figure 3a The "call noise reduction mode" in the middle will then jump to such as Figure 11 The detailed function interface shown includes a switch for "Reset and update every time you make a voice call". If the user selects to turn on the switch, the target voiceprint template will be learned once every time you make a voice call. If the switch is not turned on, the target voiceprint template will be learned once every preset period by default.

[0229] In the above embodiments, a voiceprint self-learning model was used in the target voiceprint template learning process, and a voiceprint denoising model was used in the voiceprint denoising process. In this embodiment, a joint training method can be used to train the voiceprint self-learning model and the voiceprint denoising model. The voiceprint self-learning model has been pre-trained before joint training. Figure 12As shown, the sample mixed audio can be used as the registered speech input for the voiceprint self-learning model. After the voiceprint self-learning model extracts voiceprint information, the classifier outputs the voiceprint information of the target speaker. Then, the cross-entropy loss function between the speaker label of the sample mixed audio and the voiceprint information of the target speaker is calculated.

[0230] Furthermore, the sample mixed audio is also input into the voiceprint denoising model, and the voiceprint information output by the voiceprint self-learning model will be used as a voiceprint template input into the voiceprint denoising model. After the voiceprint denoising model outputs a mask, the mask output by the voiceprint denoising model is multiplied with the frequency domain amplitude spectrum of the mixed audio to obtain the separated voiceprint denoised speech. Then, the denoising loss function between the speaker's clean speech in the sample mixed audio and the voiceprint denoised speech is calculated.

[0231] Subsequently, the loss function of joint training is calculated using the noise reduction loss function and the cross-entropy loss function. The parameters of the voiceprint self-learning model and the voiceprint noise reduction model are then adjusted using the joint training loss function until both the voiceprint self-learning model and the voiceprint noise reduction model converge.

[0232] The loss function for joint training is:

[0233] Loss = SISNR + SNR + L asym +J2

[0234] SISNR, SNR and L asym J1 and J2 are the loss functions corresponding to the voiceprint denoising model, and J3 is the cross-entropy loss function corresponding to the voiceprint self-learning model.

[0235] If the voiceprint denoising model and the voiceprint self-learning model are trained separately, the feature mapping results of the two models for mixed audio may differ, leading to problems in the final denoising result. However, this application's embodiment uses a joint training method, maximizing the consistency of feature mapping between the voiceprint denoising model and the voiceprint self-learning model. While the voiceprint denoising model is being trained, the voiceprint self-learning model is still being optimized, allowing the embedding layer of the pre-trained voiceprint self-learning model to continue updating and becoming more compatible with the voiceprint denoising model. This results in higher consistency in the feature mapping results of the two models, thereby improving the final voiceprint denoising effect.

[0236] Furthermore, both the voiceprint denoising model and the voiceprint self-learning model are inputs to the mixed audio after front-end enhancement processing. If they are trained separately, it is difficult for the voiceprint denoising model and the voiceprint self-learning model to maintain consistency in feature mapping when the front-end enhancement processing algorithm changes. However, the embodiments of this application adopt a joint training method, which can maintain the consistency of the voiceprint denoising model and the voiceprint self-learning model in feature mapping even if the front-end enhancement processing algorithm changes. Furthermore, the voiceprint denoising model and the voiceprint self-learning model trained by the joint training method can adapt to any front-end enhancement processing algorithm, and the voiceprint denoising model and the voiceprint self-learning model have stronger portability. Therefore, the embodiments of this application can decouple the voiceprint link from the call 3A link, that is, the voiceprint denoising model and the voiceprint self-learning model included in the voiceprint link can adapt to electronic devices with various different call 3A links, and can be flexibly deployed according to the memory and computing power of the electronic device.

[0237] In this embodiment, the electronic device includes an ADSP module and an AP module. The ADSP module includes a front-end enhancement module, and the AP module includes a voiceprint self-learning module and a voiceprint noise reduction module. The voiceprint self-learning module includes a voiceprint self-learning model, and the voiceprint noise reduction module includes a voiceprint noise reduction model. The following describes the voiceprint noise reduction method provided in this embodiment from the perspective of the interaction between the modules. Figure 13 As shown, the method includes:

[0238] S1301 The front-end enhancement module performs front-end enhancement processing on the audio collected during the voice call to obtain mixed audio.

[0239] S1302, The front-end enhancement module sends mixed audio to the voiceprint self-learning module.

[0240] S1303, the voiceprint self-learning module filters the mixed audio data to obtain a set of audio segments.

[0241] In each audio segment group, the cumulative duration of the audio segments is the second preset duration. For example, each audio segment group includes four 0.5-second audio segments, with a cumulative duration of 2 seconds.

[0242] S1304, The voiceprint self-learning module uses a voiceprint self-learning model to extract the first voiceprint information of each audio segment.

[0243] S1305, The voiceprint self-learning module clusters and fuses the first voiceprint information with the voiceprint information of each speaker in the voiceprint template library, and selects the second voiceprint information that has reached the preset threshold number of clustering and fusion times, and uses the second voiceprint information as the target voiceprint template of the target speaker.

[0244] S1306, The voiceprint self-learning module sends the target speaker's voiceprint template to the voiceprint noise reduction module.

[0245] S1307, The front-end enhancement module sends mixed audio to the voiceprint noise reduction module.

[0246] S1308, the voiceprint noise reduction module uses the voiceprint noise reduction model to obtain the first mask corresponding to the mixed audio.

[0247] S1309 The voiceprint noise reduction module multiplies the frequency domain amplitude spectrum of the mixed audio with the first mask to obtain the audio after voiceprint noise reduction.

[0248] This method decouples the voiceprint self-learning module and the voiceprint noise reduction module from the front-end enhancement module, allowing for flexible deployment based on computational power and functional requirements. Furthermore, the effectiveness of the voiceprint self-learning and noise reduction modules is independent of the front-end enhancement algorithm, demonstrating strong adaptability. By learning the target speaker's voiceprint template during the call, it ensures that the learned template accurately reflects the target speaker's voice during the current call, resulting in more accurate noise reduction. This allows the denoised audio to retain only the target speaker's voice, improving noise reduction performance and enhancing the user's call experience.

[0249] In a specific implementation, this application also provides a computer storage medium, including a computer program, wherein when the computer program runs on an electronic device, it causes the electronic device to execute some or all of the steps in the above embodiments. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0250] In a specific implementation, this application also provides a computer program product, which includes computer program code. When the computer program code is run on an electronic device, it causes the electronic device to perform some or all of the steps in the above method embodiments.

[0251] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0252] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0253] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0254] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A voiceprint noise reduction method, characterized in that, Applied to electronic devices, the method includes: Receive the user's first action to initiate a voice call; In response to the first operation, a voice call is initiated; The audio collected during a voice call is enhanced to obtain mixed audio. The mixed audio is subjected to voiceprint self-learning through a voiceprint self-learning model, which is a trained neural network model for learning voiceprint templates. If the duration of the voice call reaches a first preset duration and no target speaker's voiceprint template is learned, then the mixed audio is subjected to NN noise reduction to obtain a first NN noise reduction result; the mixed audio and the initial voiceprint template are input into the voiceprint noise reduction model to obtain a first mask corresponding to the mixed audio; the frequency domain amplitude spectrum of the mixed audio is multiplied by the first mask to obtain the audio after voiceprint noise reduction; wherein, the voiceprint noise reduction model is a trained neural network model for voiceprint noise reduction, and the initial voiceprint template is the optimal voiceprint information of the target speaker currently learned by the voiceprint self-learning model; The audio after voiceprint noise reduction is fused with the noise reduction result of the first NN to obtain the first audio. The first audio is transmitted to the electronic device at the other end of the voice call.

2. The method according to claim 1, characterized in that, After performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio, the method further includes: If the duration of the voice call does not reach the first preset duration, then the mixed audio is subjected to NN noise reduction to obtain a second NN noise reduction result; The mixed audio and the second NN noise reduction result are fused to obtain the second audio; The second audio is transmitted to the electronic device at the other end of the voice call.

3. The method according to claim 1, characterized in that, After performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio, the method further includes: If the target speaker's target voiceprint template has been learned, then the mixed audio and the target voiceprint template are input into the voiceprint noise reduction model to obtain the second mask corresponding to the mixed audio; Multiply the frequency domain amplitude spectrum of the mixed audio with the second mask to obtain the third audio; The third audio is transmitted to the electronic device at the other end of the voice call.

4. The method according to claim 1, characterized in that, The process of performing voiceprint self-learning on the mixed audio using a voiceprint self-learning model includes: The mixed audio that meets the speech quality requirements is processed by VAD to obtain multiple audio segments; Each time an audio segment of a cumulative second preset duration is obtained, the audio segment of the cumulative second preset duration is input into the voiceprint self-learning model to obtain the first voiceprint information output by the voiceprint self-learning model; The first voiceprint information is clustered and fused with the voiceprint information of each speaker in the voiceprint template library; When the number of clustering and fusion of the second voiceprint information with the most clustering and fusion times in the voiceprint template library reaches a preset threshold, the second voiceprint information is used as the target voiceprint template of the target speaker.

5. The method according to claim 4, characterized in that, The voiceprint template library includes the voiceprint information of each speaker during the voice call and the number of clustering and fusion of each voiceprint information. The initial voiceprint template is the voiceprint information with the most current clustering and fusion times in the voiceprint template library. The step of clustering and fusing the first voiceprint information with the voiceprint information of each speaker in the voiceprint template library includes: Calculate the similarity between the first voiceprint information and the voiceprint information of each speaker in the voiceprint template library; If the similarity between the first voiceprint information and the third voiceprint information in the voiceprint template library is greater than a preset similarity threshold, then the first voiceprint information and the third voiceprint information are clustered and fused, and the clustering and fusion count of the third voiceprint information is incremented by 1.

6. The method according to claim 4, characterized in that, The frequency domain amplitude spectrum of the first audio is the sum of the frequency domain amplitude spectrum of the audio after voiceprint noise reduction and the frequency domain amplitude spectrum of the first NN noise reduction result. Wherein, the weight of the frequency domain amplitude spectrum of the audio after voiceprint noise reduction is a dynamic smoothing factor, and the weight of the frequency domain amplitude spectrum of the first NN noise reduction result is the difference between 1 and the dynamic smoothing factor. The dynamic smoothing factor is positively correlated with the current maximum number of cluster fusions, which is the number of cluster fusions of the voiceprint information with the highest number of current cluster fusions in the voiceprint template library.

7. The method according to claim 6, characterized in that, The frequency domain amplitude spectrum of the first audio audio for: ； Wherein, S2 is the frequency domain amplitude spectrum of the audio after voiceprint denoising, and S3 is the frequency domain amplitude spectrum of the first NN denoising result. It is a dynamic smoothing factor; ； in, The preset number of times threshold, This refers to the number of times the voiceprint information with the most current clustering and fusion times is clustered in the voiceprint template library.

8. The method according to claim 2, characterized in that, The frequency domain amplitude spectrum of the second audio is the sum obtained by weighted summation of the frequency domain amplitude spectrum of the mixed audio and the frequency domain amplitude spectrum of the second NN noise reduction result; The weights of the frequency domain amplitude spectrum of the mixed audio are negatively correlated with the duration of the voice call, while the weights of the frequency domain amplitude spectrum of the second NN noise reduction result are positively correlated with the duration of the voice call.

9. The method according to claim 8, characterized in that, The frequency domain amplitude spectrum of the second audio frequency for: ; Where t0 is the time point at which the voiceprint noise reduction switch in the voice call interface is turned on after the voice call is initiated, t i t1 is the current time; t1 is the time point after the first preset duration when the voiceprint noise reduction switch is turned on; S0 is the frequency domain amplitude spectrum of the mixed audio; and S1 is the frequency domain amplitude spectrum of the second NN noise reduction result.

10. The method according to any one of claims 1-9, characterized in that, The voiceprint self-learning model and the voiceprint noise reduction model are jointly trained based on the same training set, which includes mixed audio samples after signal enhancement processing.

11. The method according to any one of claims 1-9, characterized in that, After the voice call is initiated, the method further includes: The voiceprint noise reduction switch is displayed on the call interface of the voice call; If the voiceprint noise reduction switch is in the on state, then the step of performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio is executed; If the voiceprint noise reduction switch is in the off state and a user's operation to turn on the voiceprint noise reduction switch is received, then in response to the operation, the step of performing signal enhancement processing on the audio collected during the voice call to obtain mixed audio is executed.

12. The method according to claim 11, characterized in that, The current on / off state of the voiceprint noise reduction switch is the same as the on / off state of the electronic device during the last voice communication.

13. The method according to any one of claims 1-9, characterized in that, After performing voiceprint self-learning on the mixed audio using a voiceprint self-learning model to obtain the target speaker's voiceprint template, the method further includes: The target voiceprint template is updated once every preset period; or... The target voiceprint template is updated once during each voice call.

14. The method according to any one of claims 1-9, characterized in that, The electronic device includes an ADSP module and an AP module. The ADSP module includes a front-end enhancement module, and the AP module includes the voiceprint self-learning model and the voiceprint noise reduction model. The step of performing signal enhancement processing on the audio collected during a voice call to obtain mixed audio includes: The front-end enhancement module performs signal enhancement processing on the audio collected during the voice call to obtain the mixed audio.

15. An electronic device, characterized in that, include: One or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 14.

16. A computer-readable storage medium, characterized in that, Includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 14.

Images