Speech enhancement method, device and storage medium

Abstract

The application provides a speech enhancement method, device and storage medium. The method can realize speech enhancement processing of speech data in various call modes by loading speech enhancement parameters corresponding to the call mode to the same speech enhancement model and using the speech enhancement model with updated model parameters without expanding the parameter quantity and calculation quantity of the speech enhancement model.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a speech enhancement method, device and storage medium. Background Technology

[0002] The call function is one of the most frequently used functions on mobile devices, enabling users to communicate via voice anytime, anywhere. However, due to factors such as environment, network quality, and call volume, various noise interferences often occur during calls.

[0003] Therefore, it is particularly important to find a way to reduce noise in voice data across various call modes supported by mobile terminals without increasing implementation costs, and to do so without compromising noise reduction performance or the real-time nature of the call. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide a speech enhancement method, device, and storage medium, aiming to meet the noise reduction requirements of various call modes and achieve speech enhancement processing of speech data under various call modes without increasing the number of parameters and computational load of the speech enhancement model.

[0005] In a first aspect, embodiments of this application provide a voice enhancement method. The method includes: determining the current call mode when using a call function; obtaining voice enhancement parameters corresponding to the call mode, wherein the voice enhancement parameters are model parameters obtained during the training phase of a voice enhancement model based on training data constructed according to the call mode; updating the current model parameters in the pre-trained voice enhancement model to the voice enhancement parameters; and using the voice enhancement model with updated model parameters to perform voice enhancement processing on the voice data generated in the call mode.

[0006] The call mode can be determined by detecting the currently initiated call process.

[0007] The call modes can include handheld call mode, hands-free call mode, and headset call mode.

[0008] Among them, handheld call mode, hands-free call mode, and headset call mode can be further divided into uplink call mode and downlink call mode.

[0009] The uplink call modes can be further divided into VoIP (Voice over Internet Protocol) call modes based on broadband networks (VoIP broadband call mode), VoIP call modes based on narrowband networks (VoIP narrowband call mode), circuit-switched (CS) domain call modes based on broadband networks (CS broadband call mode), and CS call modes based on narrowband networks (CS narrowband call mode).

[0010] Downlink call modes can include VoIP broadband call mode, VoIP narrowband call mode, CS broadband call mode, CS narrowband call mode, and satellite call mode based on satellite.

[0011] The training data consists of both clean and noisy speech data.

[0012] The speech enhancement parameters are the current model parameters of the network model when iteratively training the network model based on the training data until the input target data (the speech data to be enhanced) is reached. In this case, the network model is the pre-trained speech enhancement model.

[0013] For details regarding the training phase of the speech enhancement model, please refer to [link / reference]. Figures 8 to 13 The description of the embodiments will not be repeated here.

[0014] Therefore, without increasing the number of parameters and computational load of the speech enhancement model, for speech data under different call modes, it is only necessary to load the speech enhancement parameters corresponding to the call mode into the same speech enhancement model, and then use the speech enhancement model with updated model parameters to achieve speech enhancement processing for speech data under various call modes.

[0015] According to the first aspect, the speech enhancement model with updated model parameters is used to perform speech enhancement processing on the speech data generated in the call mode, including: performing time-frequency conversion processing on the speech data in the time domain generated in the call mode to obtain speech data in the frequency domain; and performing speech enhancement processing on the speech data in the frequency domain using the speech enhancement model with updated model parameters.

[0016] Among them, the operation of time-frequency conversion processing of voice data in the time domain generated in the call mode is, for example, time-frequency conversion processing based on short-time Fourier transform.

[0017] Among them, speech data in the time domain can be called time-domain data, which is the number of sample points of the collected speech data in the time domain.

[0018] Among them, speech data in the frequency domain can be called frequency domain data, or STFT window length, or input data spectrum length, etc., which means the number of sample points of the speech data after frequency conversion in the frequency domain (frequency domain sample number).

[0019] According to the first aspect, or any implementation of the first aspect above, before performing time-frequency conversion processing on the time-domain voice data generated in the call mode, the method further includes: determining whether the sampling rate of the time-domain voice data is greater than a preset sampling rate; if the sampling rate of the time-domain voice data is greater than the preset sampling rate, performing downsampling processing on the time-domain voice data to reduce the sampling rate of the time-domain voice data to the preset sampling rate.

[0020] The preset sampling rate can be understood as the selected standard sampling rate, such as 32ksps.

[0021] Therefore, by downsampling the sampling rate of speech data with a sampling rate greater than the preset sampling rate to the preset sampling rate, the length of the speech data in the frequency domain obtained by subsequent time-frequency conversion processing can be equal to the standard length. This achieves the unification of the length of the input data to the speech enhancement model, enabling speech data with different sampling rates to be processed by the speech enhancement model trained on the same network model.

[0022] According to the first aspect, or any implementation of the first aspect above, the speech enhancement model with updated model parameters is used to perform speech enhancement processing on the speech data in the frequency domain, including: determining whether the sampling rate of the speech data in the frequency domain is less than a preset sampling rate; if the sampling rate of the speech data in the frequency domain is less than the preset sampling rate, padding the speech data in the frequency domain to a standard length, the standard length being the length of the speech data in the frequency domain corresponding to the preset sampling rate; using the speech enhancement model with updated model parameters to perform speech enhancement processing on the speech data in the frequency domain whose length has been padded to the standard length; if the sampling rate of the speech data in the frequency domain is equal to the preset sampling rate, using the speech enhancement model with updated model parameters to perform speech enhancement processing on the speech data in the frequency domain.

[0023] The padding process for frequency domain speech data can be understood as normalizing the frequency domain data. Specifically, it involves standardizing the length of the input data.

[0024] In the case of a preset sampling rate of 32ksps, the standard length is the effective frequency domain data length corresponding to 32ksps.

[0025] Therefore, after time-frequency conversion, by normalizing the frequency domain data, the spectrum length of the input data (the length of the speech data in the frequency domain) under different sampling rates is unified, so that speech data with different sampling rates can be processed for speech enhancement based on the speech enhancement model trained by the same network model.

[0026] According to the first aspect, or any implementation of the first aspect above, the speech data in the frequency domain is padded to the standard length, including: padding the high-order bits of the speech data in the frequency domain with 0, and padding the length of the speech data in the frequency domain to the standard length.

[0027] For example, the length of the frequency domain speech data can also be padded to a standard length by padding with other constants. For instance, it can be padded according to a set minimum volume level, such as the value corresponding to a volume level of -50dB or -100dB.

[0028] Therefore, by padding the speech data in the frequency domain, the length of the speech data in the frequency domain is filled to the standard length, and the same input length can be achieved without resampling, which effectively saves the amount of computation.

[0029] According to the first aspect, or any implementation of the first aspect above, when using the call function, determine the current call mode, including: when using the call function, determine whether the voice enhancement function is currently enabled; if the voice enhancement function is currently enabled, determine the current call mode.

[0030] Therefore, based on user needs, it can be decided whether to perform voice enhancement processing on the voice data generated in the current call mode.

[0031] According to the first aspect, or any implementation of the first aspect above, the method further includes: providing a voice enhancement function entry point, wherein the state of the voice enhancement function entry point is a first state, and the first state indicates that voice enhancement is not currently supported; upon receiving a user operation on the voice enhancement function entry point in the first state, updating the state of the voice enhancement function entry point from the first state to a second state, wherein the second state indicates that voice enhancement is currently supported; and upon receiving a user operation on the voice enhancement function entry point in the second state, updating the state of the voice enhancement function entry point from the second state back to the first state.

[0032] The voice enhancement function can be accessed through the control center interface of the terminal device or through the settings interface.

[0033] The entry point for the voice enhancement function in the first state is, for example, Figure 4Option 40a-1 shown in (1) is the second-state voice enhancement function entry point, for example, Figure 4 Option 40a-1' is shown in (2).

[0034] The entry point for the voice enhancement function in the first state is, for example, Figure 5 The switch 40b-11 shown in (1) is the voice enhancement function entry in the second state, for example, as Figure 5 The switch 40b-11' is shown in (2).

[0035] The entry point for the voice enhancement function in the first state is, for example, Figure 6 The switch 50b-1 shown in (2) is the voice enhancement function entry in the second state, for example, as... Figure 7 The switch 50b-1' is shown in (1).

[0036] According to the first aspect, or any implementation of the first aspect above, when using the call function, determining whether the voice enhancement function is currently enabled includes: when using the call function, obtaining the status information corresponding to the voice enhancement function entry; if the status information indicates that the voice enhancement function entry is in a first state, determining that the voice enhancement function is not currently enabled; if the status information indicates that the voice enhancement function entry is in a second state, determining that the voice enhancement function is currently enabled.

[0037] Therefore, by obtaining the status information corresponding to the voice enhancement function entry, or the feature support switch information, it is possible to quickly determine whether the voice enhancement function is currently activated, and then determine whether to perform voice enhancement processing on the voice data generated in the current call mode.

[0038] According to the first aspect, or any implementation of the first aspect above, the voice enhancement parameters corresponding to the call mode are obtained according to the call mode, including: when the call mode is any call mode in the whitelist call modes, the voice enhancement parameters corresponding to the call mode are obtained.

[0039] The whitelist call mode is a call mode that supports voice enhancement processing.

[0040] For example, the call mode in the whitelist call mode can be set when the user enables the voice enhancement function, or it can be a preset mode.

[0041] For example, a blacklist call mode can be set, and the system can determine whether to obtain the corresponding voice enhancement parameters and perform voice enhancement processing by detecting whether the current call mode is not a blacklist call mode.

[0042] Therefore, based on the whitelist or blacklist mechanism, it can be determined whether to perform voice enhancement processing on the current call mode, which can better meet user needs.

[0043] According to the first aspect, or any implementation of the first aspect above, the network model used in the speech enhancement model training stage of the pre-trained speech enhancement model is the compressed network model.

[0044] The compressed network model refers to a network model from which unnecessary network structures, parameters, and other information for speech enhancement have been removed. In other words, the network model is pruned and its parameters are quantized to achieve compression.

[0045] Therefore, by using a compressed network model to train the speech enhancement model, the size of the trained speech enhancement model can be effectively reduced, thereby reducing the occupation of terminal device memory and computing resources (especially DSP memory and computing resources).

[0046] According to the first aspect, or any implementation of the first aspect above, the compressed network model is a network structure including convolutional upgrade network layers, long short-term memory network layers, and fully connected network layers.

[0047] According to the first aspect, or any implementation of the first aspect above, the voice enhancement parameters corresponding to different call modes are training data constructed based on the corresponding call modes, and different model parameters are obtained based on the same network model during the voice enhancement model training phase.

[0048] In this way, based on the original speech enhancement model corresponding to any call mode, iterative training is performed using the normalized training data corresponding to different call modes, and the speech enhancement parameters that meet the iterative requirements corresponding to different call modes are stored. This enables the trained speech enhancement model to load the speech enhancement parameters corresponding to different call modes, thereby realizing speech enhancement processing of speech data generated under different call modes.

[0049] Secondly, embodiments of this application provide a terminal device. The terminal device includes: a memory and a processor, coupled together; the memory stores program instructions, which, when executed by the processor, cause the terminal device to perform the method of the first aspect or any possible implementation thereof.

[0050] Thirdly, embodiments of this application provide a computer-readable medium for storing a computer program, the computer program including instructions for performing the method in the first aspect or any possible implementation of the first aspect.

[0051] Fourthly, embodiments of this application provide a computer program including instructions for performing the method in the first aspect or any possible implementation thereof.

[0052] Fifthly, embodiments of this application provide a chip including a processing circuit and transceiver pins. The transceiver pins and the processing circuit communicate with each other via an internal connection path. The processing circuit executes the method in the first aspect or any possible implementation of the first aspect to control the receiving pin to receive signals and to control the transmitting pin to transmit signals. Attached Figure Description

[0053] Figure 1 This is a schematic diagram illustrating a VoIP narrowband call mode and a VoIP broadband call mode as examples;

[0054] Figure 2 This is a schematic diagram illustrating a CS narrowband call mode and a CS broadband call mode as examples;

[0055] Figure 3 This is a schematic diagram illustrating voice and video calls in a satellite calling mode, as an example.

[0056] Figure 4 This is an example of a user interface diagram showing how a user can set voice enhancement functions through a user entry point;

[0057] Figure 5 This is an example illustration of another user interface in which a user sets voice enhancement features through a user entry point;

[0058] Figure 6 and Figure 7 This is an example illustration of another user interface in which a user sets voice enhancement features through a user entry point;

[0059] Figure 8 This is a schematic diagram illustrating the processing steps and overall implementation logic of a speech enhancement method provided in an embodiment of this application, as exemplarily shown.

[0060] Figure 9 This is a schematic diagram illustrating the overall implementation logic of a speech enhancement method provided in an embodiment of this application, as an example.

[0061] Figure 10 This is a schematic diagram illustrating the implementation process of the speech enhancement model training phase in a speech enhancement method provided in an embodiment of this application, as an example.

[0062] Figure 11 This is a schematic diagram illustrating the uniform input data length after time-frequency conversion processing, as an example.

[0063] Figure 12 This is an example of a specific logical diagram illustrating the time-frequency conversion stage of training data and the training stage of a speech enhancement model.

[0064] Figure 13 This is an example illustration of adjusting the mask value as a location, taking into account call delay.

[0065] Figure 14 This is a schematic diagram illustrating the implementation process of the speech enhancement model operation phase in a speech enhancement method provided in an embodiment of this application, as an example.

[0066] Figure 15 This is an example of a voice enhancement method provided in an embodiment of this application, showing the noise reduction effect before and after voice enhancement processing of voice data in any call mode;

[0067] Figure 16 This is a schematic diagram of the hardware structure of a terminal device as an example.

[0068] Figure 17 This is a schematic diagram of the software structure of a terminal device as an example. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0071] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0072] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0073] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0074] With the development of terminal technology and voice call technology, more and more terminal devices, especially mobile terminals such as mobile phones, tablets, and smartwatches, have call functions (voice call function and / or video call function), so users can use different call modes.

[0075] For example, in some implementations, the terminal device supports call modes, as shown in Table 1.

[0076] Table 1 Call Mode Table

[0077]

[0078]

[0079] Referring to Table 1, the different ways of answering the call can include handheld call mode (call mode using the device's earpiece), hands-free call mode (call mode using the device's speakerphone), and headset call mode (call mode using a headset connected to the device).

[0080] Referring to Table 1, handheld call mode, hands-free call mode, and headset call mode can be further divided into uplink call mode and downlink call mode.

[0081] Referring again to Table 1, the uplink call modes can be further divided into VoIP broadband call mode, VoIP narrowband call mode, CS broadband call mode, and CS narrowband call mode.

[0082] Referring again to Table 1, downlink call modes can include VoIP broadband call mode, VoIP narrowband call mode, CS broadband call mode, CS narrowband call mode, and satellite call mode based on satellite.

[0083] Understandably, the main differences between broadband and narrowband networks lie in bandwidth, transmission rate, and application scenarios. Broadband networks have wider bandwidth and faster data transmission rates, typically between Mbps (megabits per second) and Gbps (gigabits per second), and are generally suitable for scenarios with high bandwidth requirements, such as high-speed internet access and video calls (which involve both voice and video data). Narrowband networks have narrower bandwidth and slower data transmission rates, typically between a few hundred kbps (bitrate) and tens of kbps (kilobits per second), and are generally suitable for scenarios with lower data transmission requirements, such as voice calls (which only involve voice data and not video data).

[0084] For example, in some implementations, VoIP narrowband calling mode can be understood as voice calls made using audio / video calling software via cellular networks, WiFi networks, etc. The user interface after the voice call is connected, for example... Figure 1 Interface 10a is shown in (1).

[0085] For example, in some implementations, VoIP broadband calling mode can be understood as video calling using audio and video calling software via cellular networks, WiFi networks, etc. The user interface after the video call is connected, for example... Figure 1 Interface 10b is shown in (2).

[0086] For example, in some implementations, CS narrowband calling mode can be understood as a voice call made using a mobile terminal, such as the phone software that comes with the phone, through a Subscriber Identity Module (SIM) card or a Universal Subscriber Identity Module (USIM) card inserted into the phone. The user interface after the voice call is connected, for example... Figure 2 Interface 20a is shown in (1).

[0087] For example, in some implementations, CS broadband call mode can be understood as a video call using a mobile terminal, such as the phone software built into a mobile phone, a SIM card, or a USIM card. The user interface after the video call is connected, for example... Figure 2 Interface 20b is shown in (2).

[0088] For example, in some implementations, satellite call mode can also include voice calls and video calls. Unlike CS narrowband and CS broadband call modes, satellite call mode is based on a connected satellite, such as the BeiDou satellite system. The user interface after a voice call in satellite call mode is connected, for example... Figure 3Interface 30a shown in (1) is the user interface after a video call is connected in satellite call mode, for example... Figure 3 Interface 30b is shown in (2).

[0089] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0090] Therefore, by supporting multiple call modes, mobile terminals allow users to choose the appropriate call mode according to their needs and conduct voice and / or video communication anytime, anywhere.

[0091] However, due to factors such as environment, network quality, and call volume, various noises and interferences often exist during calls, making it difficult for both parties to hear the content clearly and affecting the user's call experience. Therefore, in order to improve voice quality and intelligibility (also known as speech clarity, which refers to the percentage of speech signals that a listener can understand transmitted through a certain sound transmission system), voice enhancement has become increasingly important, especially the suppression of environmental noise.

[0092] For example, some implementations propose using artificial intelligence (AI) noise reduction techniques, such as using various deep learning network models (speech enhancement models) to perform speech enhancement processing on speech data to obtain speech data for speech enhancement.

[0093] However, these AI-based noise reduction technologies typically only perform speech enhancement processing on speech data generated under a single call mode. Since speech data varies significantly across different call modes, this type of speech enhancement model, designed for a single call mode, cannot meet the needs of all call modes.

[0094] For example, some implementations propose storing the voice enhancement model corresponding to each call mode in the mobile terminal. This allows for voice enhancement processing using the voice enhancement model corresponding to the current call mode when making calls in different modes. However, this implementation places high demands on the memory and computing power resources of the digital signal processor (DSP) in the mobile terminal used to run the voice enhancement model and perform voice enhancement processing. For mobile terminals, besides the call function, many other applications (such as low-power applications) require DSP memory and resources. Therefore, current DSPs have strict limitations on the size and computing power of the voice enhancement model. Consequently, this implementation method has low feasibility due to its high implementation cost.

[0095] For example, the call modes currently supported by the device include VoIP broadband call mode with a sampling rate of 32 ksps (kilosamples per second), VoIP broadband call mode with a sampling rate of 8 ksp, VoIP broadband call mode with a sampling rate of 16 ksps, and VoIP broadband call mode with a sampling rate of 48 ksps. To enable the device to perform voice enhancement processing on the voice data generated under these four sampling rate VoIP broadband call modes, the mobile terminal needs to store a voice enhancement model suitable for the 32 ksp VoIP broadband call mode (for ease of distinction, the voice enhancement model in the current voice enhancement scheme is referred to as the original voice enhancement model; here, the original voice enhancement model suitable for the 32 ksp VoIP broadband call mode can be referred to as original voice enhancement model 1), an original voice enhancement model 2 suitable for the 8 ksp VoIP broadband call mode, an original voice enhancement model 3 suitable for the 16 ksp VoIP broadband call mode, and an original voice enhancement model 4 suitable for the 48 ksp VoIP broadband call mode.

[0096] In other words, based on the above implementation method, to perform voice enhancement processing on voice data generated under the four sampling rates of VoIP broadband call modes, four different original voice enhancement models need to be stored in the mobile terminal. If other call modes also need to be supported, the original voice enhancement models will require even more memory, meaning higher memory and computing resources are needed. This will severely limit the deployment and use of other applications on the DSP.

[0097] Furthermore, for some low-to-mid-range chip platforms and mobile terminals already on the market, after enabling voice enhancement, the ratio of the time spent processing voice data to the total duration of the processed voice data may be close to or even greater than 1. For example, processing 10 milliseconds (ms) of voice data may require an average time of 7-8 ms. However, according to the requirements of real-time calls, this ratio needs to be less than 1; the smaller the ratio, the faster the processing speed and the better the real-time performance. Therefore, continuing to occupy more memory and computing resources on the DSP will not only affect the deployment and use of other applications on the DSP but also impact call real-time performance.

[0098] It should be noted that the model size, the required increase factor, the amount of memory required, and the processing time listed in the above implementation methods are merely illustrative examples and are not intended as specific limitations.

[0099] For example, some implementations propose using AI noise reduction technology and training data (speech data) corresponding to various call modes to train a speech enhancement model that can support multiple call modes. However, since the speech enhancement model in this implementation includes speech enhancement processing steps for all call modes, when performing speech enhancement processing on the speech data for each call mode, all speech enhancement processing steps corresponding to all call modes will be executed. Therefore, the computational load is large and the real-time performance of speech enhancement is poor.

[0100] Therefore, it is particularly important to find a way to reduce noise in voice data across various call modes supported by mobile terminals without increasing implementation costs, and to do so without compromising noise reduction performance or the real-time nature of the call.

[0101] In view of this, this application proposes a full-link AI noise reduction method, which is a speech enhancement method for all call modes supported by mobile terminals, such as the speech data generated under all call modes listed in Table 1.

[0102] For example, in some implementations, it can be assumed that when the mobile terminal is using the call function, it will always use the voice enhancement scheme provided in the embodiments of this application to enhance the voice data generated during the call. In this case, the voice enhancement method provided in the embodiments of this application may include a voice enhancement model training phase (phase A) and a voice enhancement model running phase (phase B).

[0103] For example, in some other implementations, a user entry point can be provided, allowing users to actively set whether to enhance the voice data generated during a call, and which call modes' voice data should be enhanced. In this case, the voice enhancement method provided in this application embodiment may include a voice enhancement model training phase (phase A), a phase where the user sets whether the voice enhancement function is active (phase C), and a voice enhancement model running phase (phase B).

[0104] The following combination Figures 4 to 7 This section explains how the user can set whether the voice enhancement function is active through the user interface.

[0105] See Figure 4 In section (1), taking a mobile phone as an example, a control center interface (interface 40a) of a mobile phone is shown.

[0106] See also Figure 4In the example (1), the interface 40a may include multiple function options, such as a function option (option 40a-1) for setting whether the voice enhancement function is effective, a function option for setting whether WLAN is enabled, a function option for setting the ringing, etc.

[0107] Understandably, in some implementations, the function option used to set whether the voice enhancement function is enabled is as follows: Figure 4 In the case of option 40a-1 shown in (1), the voice enhancement function is not enabled, i.e., the voice enhancement function is not turned on. In this case, the voice data generated in the current voice call mode will not be processed for voice enhancement when the user makes any call using the mobile phone.

[0108] See also Figure 4 In example (1), when a user clicks option 40a-1, the phone responds to the user's action by enabling the voice enhancement function and refreshing the function option used to set whether the voice enhancement function is active from the style of option 40a-1 to... Figure 4 The style of option 40a-1' in interface 40a' shown in (2) is shown.

[0109] Understandably, in some implementations, the function option used to set whether the voice enhancement function is enabled is as follows: Figure 4 In the case of option 40a-1' shown in (2), the voice enhancement function is activated, i.e., the voice enhancement function is enabled. In this case, in some implementations, the voice data generated in the current voice call mode will be processed for voice enhancement when the user makes any call using the mobile phone.

[0110] Furthermore, it should be noted that when the user interface is interface 40a', after the user clicks option 40a-1', the phone will respond to the user's action by disabling the voice enhancement function and refreshing the function options used to set whether the voice enhancement function is enabled from the style of option 40a-1'. Figure 4 The style of option 40a-1 in interface 40a shown in (1) is shown.

[0111] Therefore, by providing a user entry point in the control center for setting whether the voice enhancement function is enabled or disabled, users can easily turn the voice enhancement function on or off with a single click.

[0112] See Figure 5 In section (1), taking the mobile terminal as a mobile phone as an example, another type of mobile phone control center interface (interface 40b) is shown.

[0113] For example, interface 40b may include multiple function option windows, and each function option window may include one or more function options.

[0114] See also Figure 5 In (1), for example, in some implementations, interface 40b may include a voice enhancement settings window 40b-1. The noise reduction settings window 40b-1 may include a voice enhancement switch for the user to enable or disable the voice enhancement function.

[0115] Understandably, in some implementations, the voice enhancement switch is... Figure 5 In the case of the switch 40b-11 shown in (1), the voice enhancement function is not active, that is, the voice enhancement function is not enabled. In this case, the voice data generated in the current voice call mode will not be enhanced when the user makes any call using the mobile phone.

[0116] See also Figure 5 In example (1), when a user clicks switch 40b-11, the phone responds to the user's operation by enabling the voice enhancement function and refreshing the voice enhancement switch from the style of switch 40b-11 to... Figure 5 The style of switch 40b-11' in the voice enhancement settings window 40b-1' shown in (2) is shown.

[0117] Understandably, in some implementations, the voice enhancement switch is... Figure 5 In the case of the switch 40b-11' shown in (2), the voice enhancement function is activated, i.e., the voice enhancement function is turned on. In this case, in some implementations, the voice data generated in the current voice call mode will be processed by voice enhancement when the user uses the mobile phone to make any call. In other implementations, when the voice enhancement function is turned on, the voice enhancement setting window 40b-1' can also display call modes that support voice enhancement using the voice enhancement method provided in the embodiments of this application. In this way, when the user selects any one or more call modes displayed in the voice enhancement setting window 40b-1', the mobile phone responds to the user's operation and can determine the call mode selected by the user as the call mode that needs to be voice enhanced by the voice enhancement method provided in the embodiments of this application. That is, when making a voice call in the call mode selected by the user, the voice data generated in the call mode can be voice enhanced.

[0118] For example, in some implementations, the call modes displayed in the voice enhancement settings window 40b-1', such as call mode 1 to call mode n, can be any of the call modes shown in Table 1.

[0119] For example, in some other implementations, the call modes displayed in the voice enhancement settings window 40b-1' can also be categorized into broad call modes such as handheld call mode, hands-free call mode, headset call mode, or uplink call mode, downlink call mode, etc. This way, after the user selects any one or more broad call modes, voice enhancement can be applied to all call modes within that broad call mode.

[0120] Furthermore, it should be noted that when the user interface is interface 40b', after the user clicks switch 40b-11', the phone responds to the user's action by disabling the voice enhancement function and refreshing the voice enhancement switch from the style of switch 40b-11' to... Figure 5 The style of switch 40b-11 in interface 40b shown in (1) is shown.

[0121] Therefore, by providing a user entry point in the control center to set whether the voice enhancement function is active, and offering multiple call mode settings for users to choose from, users can not only easily turn the voice enhancement function on or off with one click, but also set the call mode that needs voice enhancement according to their actual usage needs, further improving the user experience.

[0122] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0123] Furthermore, it should be understood that interface 40a, or interface 40a', or interface 40b, or interface 40b' can be any user interface on the phone that the phone switches to in response to the user's action after the user makes a specific gesture (swipe down from the top of the phone screen) in a specific area.

[0124] See Figure 6 In section (1), taking a mobile phone as an example, a mobile phone settings interface (interface 50a) is shown.

[0125] For example, the interface 50a may include multiple setting options, such as voice enhancement setting option 50a-1, WLAN setting option, Bluetooth setting option, mobile network setting option, etc.

[0126] For example, when a user taps the voice enhancement settings option 50a-1, the phone responds to the user's action, and the user interface can switch from interface 50a to... Figure 6 Interface 50b is shown in (2).

[0127] See Figure 6In example (2), interface 50b may include a voice enhancement switch for turning the voice enhancement function on or off. This embodiment assumes that the voice enhancement function is not enabled by default, i.e., the voice enhancement switch displayed on interface 50b is... Figure 6 The style of switch 50b-1 shown in (1) is an example.

[0128] For example, when a user clicks switch 50b-1, the phone responds to the user's action by activating the voice enhancement function and refreshing the user interface from interface 50b. Figure 7 Interface 50b' is shown in (1).

[0129] See Figure 7 In example (1), the style of the voice enhancement switch in interface 50b' will switch from the style of switch 50b-1 to the style of switch 50b-1'. ​​Furthermore, interface 50b' also displays a call mode list 50b-2. The call mode list 50b-2 displays call modes that support voice enhancement using the voice enhancement method provided in this application embodiment, as well as a switch corresponding to each call mode and a switch for selecting or deselecting all call modes.

[0130] For example, in some implementations, after the voice enhancement function is enabled by default, all switches displayed in the call mode list 50b-2 are in the [position not specified]. Figure 7 The pattern shown in (1) is used to indicate that voice enhancement is performed in all call modes.

[0131] For example, in the case of interface 50b', the user can switch the call mode with voice enhancement enabled in the call mode list 50b-2 to the off state as needed, such as... Figure 7 In the interface 50b shown in (2), the call mode list 50b-2' will switch the switch that indicates voice enhancement for all call modes to the following: Figure 7 The pattern shown in (2) indicates that the phone, in response to the user's operation, will switch all the switches corresponding to the call modes to the off state, and then switch the switches corresponding to the desired call modes to the on state as needed, such as switching the switches corresponding to call mode 1 and call mode 3 to the on state. Figure 7 (The style shown in (2)).

[0132] Therefore, the settings interface provides a user entry point for setting whether the voice enhancement function is enabled, as well as multiple call mode settings for users to choose from, so that users can set the call mode that needs voice enhancement according to their actual usage needs.

[0133] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0134] Furthermore, it should be noted that in some implementations, stage C can be considered as a step to be executed in stage B. To better understand the speech enhancement method provided in this application's embodiments, we will take the implementation where stage C is considered a step to be executed in stage B as an example, combined with... Figure 8 The processing steps and overall implementation logic of the speech enhancement method provided in the embodiments of this application will be described.

[0135] See Figure 8 For example, the speech enhancement model training phase may include a call pattern determination stage, a training data construction stage corresponding to the call pattern stage, a training data time-frequency conversion stage, and a speech enhancement model training stage. For specific implementation details of each stage in the speech enhancement model training phase, please refer to the following... Figures 10 to 12 The description of the illustrated embodiment will not be repeated here.

[0136] See also Figure 8 For example, after obtaining the trained voice enhancement model through the voice enhancement model training phase, the voice enhancement parameters loaded by the trained voice enhancement model (the model parameters that the voice enhancement model needs to load when performing voice enhancement processing on voice data) can be added to the voice enhancement parameter library under a specified path on the mobile terminal for management using training data corresponding to different call modes.

[0137] See also Figure 8 For example, the voice enhancement model's execution phase may include the following steps: enabling voice enhancement, determining the current call mode, loading the voice enhancement parameters corresponding to the current call mode, and performing voice enhancement based on the voice enhancement model with the loaded parameters. For specific implementation details of each step in the voice enhancement model's training phase, please refer to the following... Figure 14 The description of the illustrated embodiment will not be repeated here.

[0138] See also Figure 8 For example, after obtaining the trained speech enhancement model using the training data corresponding to each call mode, the speech enhancement model needs to be run in the DSP. In this way, after obtaining the speech data (before noise reduction) during the speech enhancement model running stage, the speech data before noise reduction can be processed according to the steps included in the speech enhancement model running stage, and then the speech data after speech enhancement can be obtained.

[0139] about Figure 8 The overall implementation logic of the speech enhancement method shown can also be implemented in some ways as follows: Figure 9 As shown. Among them, Figure 8 The training phase of the speech enhancement model includes the call pattern determination stage, the training data construction stage corresponding to the call pattern, the time-frequency conversion stage of the training data, the speech enhancement model training stage, and the stage of adding the speech enhancement parameters corresponding to different call patterns to the speech enhancement parameter library, which are respectively related to... Figure 9 The training phase of the Chinese speech enhancement model includes call patterns, training data, time-frequency conversion, neural networks, and a corresponding speech enhancement parameter library. Figure 8 The speech enhancement model operation phase includes the following steps: enabling speech enhancement, determining the current call mode, loading the speech enhancement parameters corresponding to the current call mode, and performing speech enhancement based on the speech enhancement model after loading the speech enhancement parameters. These steps are respectively related to... Figure 9 The speech enhancement model's operation phase includes speech enhancement switching, call mode, loading speech enhancement parameters corresponding to the current call mode, time-frequency conversion, and neural network implementation. For detailed implementation information on these stages, please refer to the following... Figures 10 to 14 The description of the illustrated embodiment will not be repeated here.

[0140] In other words, the voice enhancement method provided in this application embodiment can perform voice enhancement processing on voice data generated in all call modes supported by the mobile terminal without increasing the number of voice enhancement parameters and computing power in the voice enhancement model.

[0141] The following combination Figures 10 to 12 This section provides a detailed explanation of the steps involved in the training phase of the speech enhancement model.

[0142] See Figure 10 This example illustrates each stage involved in the training phase of a speech enhancement model, as well as the specific implementation steps for each stage.

[0143] S101, Confirm call mode.

[0144] For example, taking the voice enhancement model training phase as being completed on a cloud server, the operation of determining the call mode in the call mode determination phase can be determined by technicians based on the capabilities supported by the mobile terminal.

[0145] The division of call modes can be found in Table 1 and the description of the above embodiments, and will not be repeated here.

[0146] S102, Based on the call mode, construct the training data corresponding to the call mode.

[0147] The training data can be understood as including clean speech data (speech data without noise) generated in the current call mode and noisy data containing various types of noise. In other words, the training data is speech data that combines clean speech data and noisy data.

[0148] For example, in some implementations, the clean speech data included in the training data can be speech data from different groups of people (different ages, different genders, different languages), with different volumes and different durations.

[0149] For example, in some implementations, noise data may include voice data such as music, air conditioning, wind, rain, water flow, thunder, impact sounds (such as the sound of tapping on a table, car sounds, road traffic sounds, keyboard sounds, etc.), white noise, and animal sounds.

[0150] In this way, training data corresponding to each call mode can be obtained, thereby ensuring that the speech enhancement parameters obtained by training based on the training data corresponding to different call modes can effectively remove noise in the speech data generated in the corresponding call mode when loaded and used in the subsequent speech enhancement model running stage.

[0151] It should be noted that in speech processing, the speech signal is usually divided into frames of fixed duration for processing. Since the sampling rate of speech data varies in different call modes, the number of sample points (time-domain sample points, hereinafter referred to as time-domain data) contained in a single frame of speech data also varies. For example, with a frame length of 10 milliseconds (ms), for a sampling rate of 8 ksps, a single frame of speech data contains 80 time-domain data points; for a sampling rate of 16 ksps, a single frame contains 160 time-domain data points; and for a sampling rate of 32 ksps, a single frame contains 320 time-domain data points.

[0152] Using three frames of speech data as input data, the time-domain data included in the three frames of speech data are 240, 480, and 960 at different sampling rates, such as 8ksps, 16ksps, and 32ksps, respectively. When time-frequency conversion processing is performed on the speech data corresponding to these three sampling rates, for example, based on short-time Fourier transform (STFT), the number of sample points in the frequency domain (hereinafter referred to as frequency domain data) corresponding to the three sampling rates of 8ksps, 16ksps, and 32ksps are 256, 512, and 1024, respectively.

[0153] Understandably, frequency domain data is determined based on time domain data, typically in the form of 2 to the power of n. The number raised to the power of n must be greater than (closest to) or equal to the time domain data. Therefore, with 240 time domain data points, the frequency domain data is 2 to the power of 8, or 256; with 480 time domain data points, the frequency domain data is 2 to the power of 9, or 512; and with 960 time domain data points, the frequency domain data is 2 to the power of 10, or 1024.

[0154] Therefore, it can be seen that the length of the input data corresponding to 3 frames of speech data, i.e., the frequency domain data (also known as the input data spectrum length or STFT window length), is different for different sampling rates. Thus, the same network model cannot be directly used for training. In view of this, in order to unify the input data length (or dimension) under various sampling rates, so that speech data at different sampling rates can be trained using the same network model and for subsequent speech enhancement processing, the sampling rate is normalized during the time-frequency conversion stage of the training data (and the speech enhancement stage involved in the speech enhancement model's operation). Specifically, the frequency domain data at different sampling rates is unified to the same length.

[0155] For example, in some implementations, a sampling rate of 32ksps can be used as the standard sampling rate. For sampling rates lower than 32ksps, the frequency domain data corresponding to these sampling rates is padded to increase the length of the frequency domain data to the length of the frequency domain data corresponding to the 32ksps sampling rate, i.e., to 1024.

[0156] For example, in some other implementations, for sampling rates higher than 32ksps, downsampling is performed first to reduce the sampling rate to 32ksps, and then subsequent processing is performed based on the frequency domain data corresponding to the downsampling. After the processing is completed, upsampling is performed.

[0157] In other words, after time-frequency conversion, the frequency domain data is normalized so that the input data (input data spectrum length) is the same as the input data spectrum length corresponding to the standard sampling rate.

[0158] See Figure 12 This example illustrates the padding operation performed on frequency domain data with sampling rates lower than the standard sampling rate (e.g., 32ksps), such as 8ksps and 16ksps. Specifically, it shows the effective frequency domain data and the padded data in the input data of uniform length after padding the high bits with zeros.

[0159] In this way, by performing the normalization process mentioned above on the sampling rate, the spectral length of the input data under different sampling rates can be unified.

[0160] Furthermore, it should be noted that in some implementations, the length of the frequency domain speech data can also be padded to the standard length by using other constants. For example, it can be padded according to the lowest set volume, such as the value corresponding to a volume of -50dB or -100dB.

[0161] Therefore, by padding the speech data in the frequency domain, the length of the speech data in the frequency domain is filled to the standard length, and the same input length can be achieved without resampling, which effectively saves the amount of computation.

[0162] Furthermore, based on formula (1), it can be seen that the spectral resolution corresponding to different sampling rates is the same.

[0163]

[0164] Wherein, “sample rate” is the sampling rate, such as 8ksps, 16ksps, 32ksps, etc. mentioned in the above embodiments; “STFT num” is the STFT window length, that is, the frequency domain data, or the length of the input data spectrum; and “△f” is the spectral resolution.

[0165] To better understand the relationship between time-domain data, frequency-domain data (STFT window length), spectral resolution, and normalized frequency-domain data (the uniform length of the input data spectrum, or STFT window length) at different sampling rates, taking a standard sampling rate of 32ksps as an example, and referring to Table 2, we illustrate the relationship between the above parameters at several common sampling rates, such as 8ksps, 16ksps, 32ksps, and 48ksps.

[0166] Table 2. Parameter Relationships at Different Sampling Rates

[0167]

[0168] Therefore, by standardizing the input data spectrum length and spectrum resolution, the computational load on the network model is effectively reduced.

[0169] Based on the above description, in the embodiments of this application, the training data time-frequency conversion step may include... Figure 10 Steps S103 to S106 are shown in the figure.

[0170] S103, whether the sampling rate of the training data is greater than the preset sampling rate.

[0171] The preset sampling rate can be understood as the standard sampling rate. To avoid excessively long input data, the standard sampling rate can be set to 32 ksps.

[0172] Thus, by determining whether the sampling rate of the training data is greater than a preset sampling rate (e.g., 32 ksps), if it is greater (e.g., the current sampling rate of the training data is 48 ksps), the training data is first downsampled to reduce its sampling rate to the preset sampling rate, i.e., step S104 is executed. Then, based on the short-time Fourier transform, the downsampled training data undergoes time-frequency conversion processing, i.e., step S105 is executed. Conversely, if the sampling rate of the training data is less than or equal to the preset sampling rate, the downsampled training data is directly converted to time-frequency based on the short-time Fourier transform, i.e., step S105 is executed.

[0173] S104 performs downsampling on the training data, reducing the sampling rate of the training data to a preset sampling rate.

[0174] For example, in some implementations, during the downsampling process of training data, the signal in the training data can first be subjected to anti-aliasing filtering and then extracted to obtain the downsampled training data.

[0175] Taking the training data corresponding to 48ksps as an example, which includes 1440 time-domain data points, then 960 time-domain data points need to be extracted from these 1440 frequency-domain data points as the time-domain data that needs to be processed for time-frequency conversion at the 48ksps sampling rate.

[0176] S105 performs time-frequency conversion processing on the training data based on short-time Fourier transform.

[0177] That is, the training data is transformed from the time domain to the frequency domain. For example, the value of n is determined based on the time domain data, and then converted into the corresponding frequency domain data according to the power of 2.

[0178] S106, fill in the frequency domain data corresponding to the training data with a sampling rate lower than the preset sampling rate, and unify the length of the training data after time-frequency conversion processing.

[0179] That is, padding is performed on the frequency domain data corresponding to training data with a sampling rate lower than the preset sampling rate, for example... Figure 11 As shown.

[0180] Thus, the time-frequency conversion of training data was completed through steps S103 to S106.

[0181] S107: After time-frequency conversion, the training data of uniform length is input into the network model for iterative training until the target data is obtained.

[0182] For example, in some implementations, the network model is, for instance, a neural network model.

[0183] For example, in some implementations, the network model is, for instance, a Convolutional Recurrent Neural Network (CRNN) model.

[0184] For example, in some implementations, the CRNN model is a network structure that includes Convolutional Neural Networks (CNN) layers, Long Short-Term Memory (LSTM) networks, and Fully Connected Network (FC) layers, such as... Figure 12 The network model shown in the figure.

[0185] based on Figure 12 The network model shown, during the speech enhancement model training phase, after constructing training data corresponding to different call modes, can first perform time-frequency conversion on the training data. See also... Figure 12 In the time-frequency conversion stage of training data, steps S103 and S104 described above can be executed first. Then, based on the short-time Fourier transform, the training data is processed by time-frequency conversion, i.e., an STFT operation is performed. Accordingly, after completing the STFT operation, step S106 can be executed as needed.

[0186] See also Figure 12 After completing the time-frequency conversion of the training data, the training data (frequency domain data) obtained in this step can be used as input data. The input network structure is a network model including CNN layers, LSTM layers and FC layers, and then the mask value corresponding to the input data is obtained.

[0187] See also Figure 12 The input data is processed by combining the Mask corresponding to the input data with the input data itself to obtain the frequency domain data after speech enhancement.

[0188] Since the speech data after speech enhancement processing is required to be time-frequency data, it is also necessary to perform an inverse short-time Fourier transform (ISTFT) on the frequency domain data after speech enhancement processing, thereby transforming the frequency domain data after speech enhancement processing into time domain data. In this way, the number of speech samples to be enhanced is obtained.

[0189] For any training data corresponding to any call mode, according to Figure 12The training process shown is iterative, continuing until the output speech data for speech enhancement is clean, or in other words, meets the noise reduction requirements. At this point, training the network model using the training data corresponding to that call mode can be stopped. In this case, the corresponding model parameters in the network model can be used as the speech enhancement parameters required for noise reduction in the speech enhancement model corresponding to that call mode.

[0190] Furthermore, it should be noted that since the final output of the network model is the estimated time-domain waveform of clean speech data (here, the clean speech data is estimated by the network model and will be referred to as the estimated time-domain waveform of the speech), and one of the important evaluation metrics for model performance is the scale-invariant source-to-noise ratio (SI-SNR), the technical solution provided in this application does not use the mean square error between the estimated time-domain waveform of the speech and the time-domain waveform of the clean speech (here, the time-domain waveform of the clean speech data in the training data). Instead, it uses SI-SNR to design a loss function to determine when to end the iterative training.

[0191] The SI-SNR loss function can be determined according to formulas (2) to (4):

[0192]

[0193]

[0194] in, Here, S is the estimated target data, S is the clean target data (clean speech data without noise fusion when constructing training data), and e is the target data. noise It's noisy data.

[0195] In addition, it should be noted that, Both S and S need to be normalized to a mean of 0 to ensure scale invariance.

[0196] In addition, it should be noted that since a larger SI-SNR value indicates better speech quality, and gradient descent is used to train the model during the training process, the actual loss function is defined as a negative SI-SNR.

[0197] In addition, it should be noted that the network model selected during the training phase of the speech enhancement model can be any original speech enhancement model corresponding to any call mode.

[0198] Therefore, based on the loss function described above, we can determine when to end the iterative training.

[0199] In addition, it should be noted that since the voice data of different call modes will be different, in order to ensure that the voice data generated in each call mode can be effectively denoised, corresponding voice enhancement methods such as signal-to-noise ratio and distortion type can be designed according to different call modes during the training phase, and then the corresponding training data can be used to iteratively train the network model.

[0200] In addition, it should be noted that in order to reduce the consumption of DSP memory and computing resources, the original speech enhancement model used can be a network model with a compressed model structure.

[0201] Understandably, the compressed network model mentioned in the embodiments of this application refers to a network model with unnecessary network structures, parameters, and other information removed for speech enhancement. This effectively reduces the size of the trained speech enhancement model, thereby reducing the consumption of DSP memory and computing resources.

[0202] S108 uses the model parameters corresponding to the network model when outputting target data as the voice enhancement parameters corresponding to the call mode.

[0203] Therefore, by using the training data corresponding to each call mode and training in the manner described above, the speech enhancement parameters for the corresponding call mode can be obtained.

[0204] For example, in a scenario where the voice enhancement model is trained by a cloud server, the voice enhancement parameters corresponding to each call mode can be sent from the cloud server to the mobile terminal.

[0205] Furthermore, it's important to note that besides the sampling rate, different call modes also have different latency requirements. For example, in uplink call modes, the signal-to-noise ratio is generally worse during voice enhancement processing, requiring more future frame information to improve noise reduction. In downlink call modes, however, the voice data being processed has already undergone voice enhancement (done during uplink), so future frame information isn't needed to enhance noise reduction. Therefore, in some implementations, different call delays can be preset for different call modes.

[0206] For example, in the uplink call mode (call mode 1) of the hands-free call mode, since the signal-to-noise ratio is the worst, the call latency can be set to 2 frames.

[0207] For example, in the uplink call mode (call mode 2) of the handheld call mode, the signal-to-noise ratio is better than that of call mode 1, so the call latency of call mode 2 can be set to 1 frame.

[0208] For example, for various downlink call modes, since the input voice data has already undergone voice enhancement processing at the other end, the latency can be set to 0 frames.

[0209] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0210] Based on the set call delay and the following formulas (5) and (6), the position of the Mask input during the training and running phases can be changed, thereby enabling the voice enhancement scheme provided in this application embodiment to be better suited to actual usage needs.

[0211] mask[i]=f(input[im],...,input[i]) (5)

[0212] output[i]=mask[i]*input[i-△t] (6)

[0213] Where m is the length of the input data, i is the current frame, and Δt is the call delay.

[0214] Combination Figure 13 Taking the current frame as the 8th frame and the call delay as 2 seconds as an example, the current frame is then processed using the mask[8] input by the speech enhancement model. Figure 12 The one shown During the operation, the processing is specifically performed on the input[i-△t] frame, i.e., the input[8-2] frame (the 6th frame), rather than on the current frame, i.e., the 8th frame. That is, mask[8] is applied to input[6]. In this way, by changing the position of the mask, i.e., the position of mask[i], speech enhancement processing of speech data under different call delays can be achieved.

[0215] Thus, the processing of the speech enhancement model training stage is realized. In the speech enhancement model training stage provided in this application embodiment, based on the original speech enhancement model corresponding to any call mode, iterative training is performed using normalized training data corresponding to different call modes, and the speech enhancement parameters that meet the iterative requirements corresponding to different call modes are stored, so that the trained speech enhancement model can load the speech enhancement parameters corresponding to different call modes, and realize speech enhancement processing of speech data generated under different call modes.

[0216] The following combination Figure 14 This section provides a detailed explanation of the stages involved in the operation of the speech enhancement model.

[0217] See Figure 14This example illustrates each stage involved in the operation of the speech enhancement model, as well as the specific implementation steps for each stage.

[0218] S201, when using the call function, determine whether the voice enhancement function is enabled.

[0219] Whether a call function is in use can be determined by detecting whether a call process is currently initiated. That is, if a call process is initiated, it can be determined that the call function is currently in use.

[0220] For example, in some implementations, the mobile terminal can obtain whitelist / blacklist information recorded in the whitelist / blacklist parsing module of the audio manager, as well as feature support switch information, through the audio control center.

[0221] In this application embodiment, the whitelist call mode can be understood as a call mode that supports the use of the voice enhancement method provided in this application embodiment. Correspondingly, the blacklist call mode can be understood as a call mode that does not support the use of the voice enhancement method provided in this application embodiment.

[0222] The feature support switch information can be understood as a description of the feature support switch information corresponding to each call mode, indicating whether the voice enhancement function is enabled or disabled in the current call mode.

[0223] Understandably, whitelist call mode, blacklist call model, and feature support on / off information can be configured according to... Figures 4 to 8 The system provides information on enabling or disabling the voice function and selecting the call mode, which is then saved in a specified path on the mobile terminal. Therefore, when using the call function, this information can be retrieved from the specified path to determine whether the voice function is currently enabled.

[0224] For example, if it is determined that the voice enhancement function is enabled, step S202 is executed; otherwise, the voice data is not processed by the voice enhancement method provided in this application embodiment, and it is directly parsed and played, or recorded and sent.

[0225] S202, Determine the current call mode.

[0226] Understandably, different call modes can be implemented through different call processes (which can be from different applications or from the same application). Therefore, by obtaining information about the call process, the current call mode can be determined.

[0227] S203: Based on the determined call mode, obtain the corresponding voice enhancement parameters.

[0228] Understandably, during the training phase of the speech enhancement model, when storing the speech enhancement parameters for different call modes, a correspondence between the call modes and the speech enhancement parameters can be established. Thus, based on this correspondence, given the current call mode, the speech enhancement parameters matching the current call mode can be quickly retrieved from the speech enhancement parameter library.

[0229] S204, determine whether the sampling rate of the voice data generated in the current call mode is greater than the preset sampling rate.

[0230] Understandably, the speech data here refers to the speech data before speech enhancement.

[0231] Specifically, if the sampling rate of the voice data generated in the current call mode is greater than the preset sampling rate, step S205 is executed; otherwise, if the sampling rate of the voice data generated in the current call mode is less than or equal to the preset sampling rate, step S206 is executed.

[0232] S205 performs downsampling processing on the voice data, reducing the sampling rate of the voice data to a preset sampling rate.

[0233] S206 performs time-frequency conversion processing on speech data based on short-time Fourier transform.

[0234] S207, fill in the frequency domain data corresponding to the speech data with a sampling rate lower than the preset sampling rate, and unify the length of the speech data after time-frequency conversion processing.

[0235] The operations of steps S204 to S207 are largely the same as those of steps S103 to S106 in the above embodiments. For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0236] S208, Update the current model parameters of the speech enhancement model using the obtained speech enhancement parameters.

[0237] That is, the speech enhancement parameters obtained from the speech enhancement parameter library are used as the model parameters of the speech enhancement model.

[0238] S209, using the updated speech enhancement model with speech enhancement parameters, performs speech enhancement processing on the speech data after time-frequency conversion to obtain speech-enhanced speech data.

[0239] Therefore, without increasing the number of parameters and computational load of the speech enhancement model, the noise reduction requirements of various call modes are met, and speech enhancement processing of speech data under various call modes is achieved. The real-time performance of speech enhancement is guaranteed without reducing the noise reduction performance.

[0240] Taking the original noise reduction model corresponding to the VoIP broadband call mode in the uplink call mode as an example, the network model used in the training stage of the speech enhancement model in this application embodiment is used as an example. In the running stage, by loading the speech enhancement parameters corresponding to different call modes into the speech enhancement model trained according to the above embodiment, speech enhancement processing of speech data generated in all call modes supported by the mobile terminal can be realized. For example, speech enhancement processing can also be performed well on speech data generated in the CS narrowband call mode in the uplink call mode. The spectrograms before and after speech enhancement are as follows: Figure 15 As shown in (1) and (2). However, the current solution uses the original noise reduction model corresponding to the VoIP broadband call mode in the uplink call mode, which can only perform speech enhancement processing on the speech data generated in the VoIP broadband call mode. It cannot perform speech enhancement processing on speech data generated in other call modes. For example, for the speech data generated in the CS narrowband call mode in the uplink call mode, the spectrograms before and after using the original speech enhancement model for speech enhancement processing may be completely different. Figure 15 As shown in (1), the noise reduction effect is poor, or it is impossible to reduce the noise of voice data generated in the CS narrowband call mode in the uplink call mode.

[0241] The specific voice enhancement models and the voice enhancement models provided in this application support the processing of voice data generated under which call modes, as shown in Table 3.

[0242] Table 3. Speech data processing for each call mode by the speech enhancement model.

[0243]

[0244]

[0245] It should be understood that Table 3 is merely an example of the original noise reduction model corresponding to the VoIP broadband call mode in the uplink call mode used in the training stage of the speech enhancement model in this application embodiment. It is only an example listed for better understanding of the technical solution of this embodiment and is not intended as the only limitation of this embodiment.

[0246] Therefore, compared to some implementations where different call modes correspond to a separate original speech enhancement model, and where all the parameters required for all call modes are integrated into a single original speech enhancement model, the speech enhancement method provided in this application satisfies the noise reduction requirements of various call modes and achieves speech enhancement processing of speech data under various call modes without expanding the number of parameters and computational load of the speech enhancement model.

[0247] Furthermore, it should be noted that in the technical solutions provided in the embodiments of this application, speech enhancement may include enhancement of speech quality and / or enhancement of intelligibility.

[0248] The improvement in voice quality can be reflected in many aspects such as voice noise reduction, latency, and volume, and this application does not impose any restrictions on these aspects.

[0249] Furthermore, it should be noted that the voice enhancement method provided in the above embodiments can be applied to terminal devices, such as mobile terminals with call functions, such as mobile phones, tablets, and smartwatches.

[0250] It is understood that, in order to achieve the above-mentioned functions, the terminal device includes hardware and / or software modules corresponding to perform each function. Based on the algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0251] For ease of explanation, this application uses a mobile phone as an example to illustrate the hardware structure of such terminal devices.

[0252] See Figure 16 The mobile phone 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0253] The processor 110 may include one or more processing units, such as an application processor (AP), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP) 110A, a baseband processor, a neural network processing unit (NPU) 110B, etc., which will not be listed here and this application does not limit them.

[0254] For example, in some implementations, the DSP used for speech enhancement processing of speech data can be, for example, a DSP specifically designed for speech enhancement processing of speech data (Audio DSP, ADSP).

[0255] Furthermore, it is understandable that the controller mentioned above, which serves as a processing unit, can be the nerve center and command center of the mobile phone 100. In practical applications, the controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0256] The aforementioned DSP is used to process digital signals. Specifically, in addition to processing digital image signals, the DSP can also process other digital signals. For example, when the mobile phone 100 selects a frequency point, the DSP can be used to perform Fourier transform on the frequency point energy, etc. Specifically, in the technical solution provided in the embodiments of this application, the time-frequency conversion processing performed before audio speech enhancement processing of the speech data, the processing in the speech enhancement stage, and the inverse Fourier transform processing of the content to be enhanced can all be completed by the DSP.

[0257] Regarding the aforementioned NPU, specifically in the technical solution provided in this application embodiment, it can be used to complete the training operation of the speech enhancement model. Understandably, in order to reduce the resource consumption of the mobile phone 100, the training of the speech enhancement model can also be completed by a server (which can be a cloud server) and then distributed to the mobile phone 100.

[0258] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0259] Furthermore, it should be noted that in some implementations, different processing units can be independent devices. That is, each processing unit can be considered a processor. In other implementations, different processing units can be integrated into one or more processors.

[0260] In addition, the processor 110 may also include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc., which will not be listed here, and this application does not impose any limitations on them.

[0261] In addition, processor 110 may also include memory for storing instructions and data. In some implementations, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has just used or is recurring. If processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of processor 110, and thus improves system efficiency.

[0262] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as a call application capable of various call modes, such as an audio / video application, a telephone application, etc.). The data storage area may store data created during the use of the mobile phone 100. In addition, the internal memory 121 may include high-speed random access memory, such as double data rate synchronous dynamic random access memory (DDR); it may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0263] Taking the internal memory 121 as DDR as an example, in the speech enhancement method provided in this application embodiment, the speech enhancement parameters obtained through the speech enhancement model training stage can be stored in DDR, that is, the speech enhancement parameter library is located in DDR.

[0264] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0265] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0266] Among them, the mobile communication module 150 can provide wireless communication solutions for mobile phone 100, including second-generation wireless telephone technology (2G), third-generation mobile communication technology (3G), fourth-generation mobile communication technology (4G), and fifth-generation mobile communication technology (5G).

[0267] The wireless communication module 160 can provide solutions for wireless communication applications on the mobile phone 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0268] based on Figure 16 The hardware structure shown allows the wireless communication function of the mobile phone 100 to be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0269] Furthermore, it should be noted that for implementations where the speech enhancement model is trained by a cloud server or other servers, the mobile phone 100 can communicate with the cloud server or other servers providing the speech enhancement model via the mobile communication module 150 or the wireless communication module 160. For example, the mobile phone 100 can send a request to the cloud server via the mobile communication module 150 to obtain or update the speech enhancement model and corresponding speech enhancement parameters. Accordingly, the cloud server can send the trained speech enhancement model and corresponding speech enhancement parameters to the mobile phone 100 based on the request.

[0270] In addition, it should be noted that in scenarios where the speech enhancement model is trained by a cloud server (or other servers), the cloud server can customize a speech enhancement model suitable for different mobile phones 100 according to the customized needs of different configurations of mobile phones 100, and update and iterate the speech enhancement model according to the speech enhancement results fed back by different mobile phones 100, and send the trained speech enhancement model and corresponding speech enhancement parameters to the mobile phone 100.

[0271] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0272] The audio module 170 may include a speaker 170A, a receiver 170B, a microphone 170C, and a headphone jack 170D. For example, the mobile phone 100 can implement audio functions through the application processor and the speaker 170A, receiver 170B, microphone 170C, and headphone jack 170D in the audio module 170. For example, voice calls in various call modes.

[0273] Specifically, in the technical solution provided in this application embodiment, the audio function can be implemented through an audio module 170 and a DSP. The audio module 170 can be used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. The DSP can be used to perform speech enhancement processing on the audio signal output by the audio module 170 when the audio noise reduction function is enabled.

[0274] That concludes the introduction to the hardware structure of the Mobile 100. It should be understood that... Figure 16 The mobile phone 100 shown is merely an example. In a specific implementation, the mobile phone 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. Figure 16 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0275] Taking a mobile phone as the terminal device for implementing the voice enhancement method provided in the embodiments of this application, the following describes the software structure of the mobile phone and the interaction between the mobile phone and the hardware during the implementation of the voice enhancement method provided in the embodiments of this application.

[0276] Before explaining the software structure of a mobile phone, we will first explain the architecture that a mobile phone software system can adopt.

[0277] Specifically, in practical applications, mobile phone software systems can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture.

[0278] Furthermore, it is understood that the software systems currently used by mainstream terminal devices include, but are not limited to, Windows, Android, and iOS systems. For ease of explanation, this application embodiment uses the layered architecture of the Android system as an example to illustrate the software structure of the mobile phone 100.

[0279] Furthermore, the speech enhancement method provided in the embodiments of this application is also applicable to other systems in specific implementations.

[0280] See Figure 17 This is a software structure block diagram of a mobile phone according to an embodiment of this application.

[0281] like Figure 17 As shown, the layered architecture of a mobile phone divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. Specifically, in this embodiment, the Android system is divided into three layers, from top to bottom: the application layer, the application framework layer, and the hardware abstraction layer (HAL).

[0282] The application layer can include a series of application packages. For example... Figure 17 As shown, the application package may include applications such as a calling application and an audio control center.

[0283] For example, in some implementations, the calling application is, for instance, an audio-visual application capable of making audio-visual calls in VoIP broadband mode and / or VoIP narrowband mode, a telephone application capable of making audio-visual calls in CS broadband mode and / or CS narrowband mode, and / or satellite mode, etc.

[0284] The audio control center is used to send control information to the audio manager (AudioFlinger) and to obtain status information from the audio service (AudioService).

[0285] For example, in some implementations, the audio control center sends control information to the audio manager, such as toggling information for setting whitelist / blacklist call modes and enabling / disabling voice enhancement features in different call modes.

[0286] In this application embodiment, the whitelist call mode can be understood as a call mode that supports the use of the voice enhancement method provided in this application embodiment. Correspondingly, the blacklist call mode can be understood as a call mode that does not support the use of the voice enhancement method provided in this application embodiment.

[0287] The feature support switch information can be understood as a description of the feature support switch information corresponding to each call mode, indicating whether the voice enhancement function is enabled or disabled in the current call mode.

[0288] For example, in some implementations, the audio control center obtains status information about the audio service, such as status information indicating that the audio service is started, or status information indicating that the audio service is destroyed.

[0289] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. In practical applications, the application layer may also include other application packages, such as settings applications, map applications, etc., which will not be listed here, and this application does not impose any limitations on them.

[0290] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications within the application layer. In some implementations, these APIs and frameworks can be described as functions. For example... Figure 17 As shown, the application framework layer includes functions such as the AudioTrack module, the AudioRecord module, the AudioService, the AudioFlinger module, the Status Awareness module, and the CameraService.

[0291] The audio resource management module can be understood as a function that manages and plays individual audio resources. Specifically, in this embodiment, the audio resource management module is used to manage and play voice data generated in uplink call mode.

[0292] The audio recording management module can be understood as a function for managing audio recording. Specifically, in this embodiment, the audio recording management module is used to manage and record voice data generated in downlink call mode.

[0293] The Audio Manager is the audio service in the Android system. It is the core of the entire audio system, responsible for managing all audio inputs and outputs, as well as audio mixing, processing, and routing. In Android, all audio applications need to access audio hardware, such as speakers and microphones, through the Audio Manager.

[0294] Audio services are primarily used to control the audio input and output process, such as adjusting volume, setting volume mode, setting Bluetooth audio mode, and setting call mode.

[0295] The status awareness module is used to sense various status information of the device. For example, it senses the usage status of the camera and then determines whether the current call scenario is video (VoIP broadband call mode, CS broadband call mode, etc.) or voice (VoIP narrowband call mode, CS narrowband call mode, etc.).

[0296] Camera service is a system-level service in Android that is primarily responsible for managing and controlling the camera hardware on the device.

[0297] Understandably, during the playback of voice data generated in uplink call mode, the audio resource management module needs to transmit the voice data to the audio manager. The whitelist / blacklist parsing model in the audio manager determines that the voice data generated in the current call mode requires voice enhancement processing. The audio manager then accesses the DSP through the audio hardware abstraction layer. The voice enhancement model and audio power amplifier algorithm in the DSP process the voice data sequentially, and finally deliver the processed voice data to the speaker for playback.

[0298] During the recording of voice data generated in downlink call mode, if the voice data generated in the current call mode needs to be enhanced, the microphone will transmit the collected voice data to the voice enhancement model in the DSP for voice enhancement processing. The voice data obtained after voice enhancement processing will be transmitted to the audio manager through the audio hardware abstraction layer, and finally fed back to the audio recording management module through the audio manager.

[0299] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. In practical applications, the application framework layer may also include other functional functions, such as content providers, resource managers, window managers, etc., which will not be listed here, and this application does not impose any limitations on them.

[0300] Here, HAL is the interface layer located between the operating system kernel and the hardware circuitry. HAL may include, for example, the Audio Hardware Abstraction Layer (Audio HAL) described in this application embodiment, as well as the Camera Hardware Abstraction Layer (Camera HAL), the Display Driver Hardware Abstraction Layer, etc. Specifically, the Audio HAL provides an interface for accessing hardware, allowing the audio manager to access the DSP and audio hardware through the interface provided by the Audio HAL.

[0301] That concludes the introduction to the phone's software architecture. As you can understand, Figure 17 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the mobile phone. In other embodiments of this application, the mobile phone may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0302] Furthermore, it should be noted that, in practical application scenarios, the voice enhancement methods provided in the above embodiments, implemented by the terminal device, can also be executed by a chip system included in the terminal device. This chip system may include a processor. The chip system can be coupled to a memory, enabling it to call computer programs stored in the memory during runtime to implement the steps executed by the terminal device. The processor in this chip system can be an application processor or a non-application processor.

[0303] In addition, this application embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a terminal device, the terminal device performs the above-mentioned related method steps to implement the speech enhancement method in the above embodiment.

[0304] In addition, this application also provides a computer program product that, when run on a terminal device, causes the terminal device to perform the above-mentioned related steps to implement the speech enhancement method in the above embodiments.

[0305] In addition, embodiments of this application also provide a chip (which may also be a component or module), the chip may include one or more processing circuits and one or more transceiver pins; wherein, the transceiver pins and the processing circuits communicate with each other through internal connection paths, the processing circuits execute the above-mentioned related method steps to implement the voice enhancement method in the above embodiments, so as to control the receiving pin to receive signals and control the transmitting pin to transmit signals.

[0306] Furthermore, as can be seen from the above description, the terminal device, computer-readable storage medium, computer program product, or chip provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0307] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A speech enhancement method, characterized in that, include: When using the call function, determine the current call mode; According to the call mode, obtain the voice enhancement parameters corresponding to the call mode. The voice enhancement parameters are model parameters obtained in the voice enhancement model training stage based on the training data constructed according to the call mode. Update the current model parameters in the pre-trained speech enhancement model to the speech enhancement parameters; The voice enhancement model, updated with model parameters, is used to perform voice enhancement processing on the voice data generated in the call mode. The process of using the updated speech enhancement model with model parameters to perform speech enhancement processing on the speech data generated in the call mode includes: The speech data in the time domain is subjected to time-frequency conversion to obtain speech data in the frequency domain; wherein, before performing time-frequency conversion, the method further includes: determining whether the sampling rate of the speech data in the time domain generated in the call mode is greater than a preset sampling rate; if the sampling rate of the speech data in the time domain is greater than the preset sampling rate, the speech data in the time domain is downsampled to reduce the sampling rate of the speech data in the time domain to the preset sampling rate. Determine whether the sampling rate of the speech data in the frequency domain is less than a preset sampling rate; If the sampling rate of the speech data in the frequency domain is less than the preset sampling rate, the speech data in the frequency domain is padded to a standard length, which is the length of the speech data in the frequency domain corresponding to the preset sampling rate; the speech enhancement model updated with model parameters is used to perform speech enhancement processing on the speech data in the frequency domain whose length is padded to the standard length. When the sampling rate of the speech data in the frequency domain is equal to the preset sampling rate, the speech enhancement model updated with model parameters is used to perform speech enhancement processing on the speech data in the frequency domain.

2. The method according to claim 1, characterized in that, The step of padding the speech data in the frequency domain to a standard length includes: The high-order bits of the speech data in the frequency domain are padded with zeros to fill the length of the speech data in the frequency domain to the standard length.

3. The method according to claim 1, characterized in that, When using the call function, determining the current call mode includes: When using the call function, determine whether the voice enhancement function is currently enabled; If the voice enhancement feature is currently enabled, determine the current call mode.

4. The method according to claim 3, characterized in that, The method further includes: A voice enhancement function entry is provided, and the state of the voice enhancement function entry is a first state, which indicates that voice enhancement is not currently supported. Upon receiving a user operation on the voice enhancement function entry in the first state, the state of the voice enhancement function entry is updated from the first state to the second state, whereby the second state indicates that voice enhancement is currently supported. Upon receiving a user operation on the voice enhancement function entry in the second state, the state of the voice enhancement function entry is updated from the second state to the first state.

5. The method according to claim 4, characterized in that, When using the call function, determining whether the voice enhancement function is currently enabled includes: When using the call function, obtain the status information corresponding to the voice enhancement function entry; If the status information indicates that the voice enhancement function entry is in the first state, it is determined that the voice enhancement function is not currently enabled; If the status information indicates that the voice enhancement function entry is in the second state, it is determined that the voice enhancement function is currently enabled.

6. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining the voice enhancement parameters corresponding to the call mode according to the call mode includes: If the call mode is any of the whitelisted call modes, obtain the voice enhancement parameters corresponding to the call mode.

7. The method according to any one of claims 1 to 5, characterized in that, The speech enhancement model obtained through pre-training uses a compressed network model during the speech enhancement model training phase.

8. The method according to claim 7, characterized in that, The compressed network model is a network structure that includes convolutional upgrade network layers, long short-term memory network layers, and fully connected network layers.

9. The method according to any one of claims 1 to 5, characterized in that, The speech enhancement parameters corresponding to different call modes are training data constructed based on the corresponding call modes. During the speech enhancement model training phase, different model parameters are obtained based on the same network model.

10. A terminal device, characterized in that, The terminal device includes: a memory and a processor, the memory and the processor being coupled; the memory stores program instructions, which, when executed by the processor, cause the terminal device to perform the speech enhancement method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The method includes a computer program that, when run on a terminal device, causes the terminal device to perform the speech enhancement method as described in any one of claims 1 to 9.

Images