Speech noise reduction method, device and electronic equipment based on artificial intelligence

Abstract

The present application provides a method, device, electronic device and computer-readable storage medium for speech noise reduction based on artificial intelligence; the method comprises: obtaining a speech signal; obtaining constraint conditions corresponding to the speech signal; calling a noise reduction model adapted to the constraint conditions to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal. Through the present application, the noise reduction effect and efficiency of the noise reduction processing can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to an artificial intelligence-based speech noise reduction method, device, equipment and computer-readable storage medium. Background Art

[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0003] In voice calls or human-computer voice interaction applications, the collected voice signals generally contain noise, which will affect the call and recognition effects. The voice signals need to be processed for noise reduction to improve the voice signal-to-noise ratio, enhance call clarity and intelligibility, and make machine voice recognition more accurate and effective. However, the efficiency of noise reduction processing for voice signals in related technologies is low, and it is impossible to effectively perform noise reduction processing on voice signals in real time. Summary of the invention

[0004] The embodiments of the present application provide a speech noise reduction method, device, electronic device and computer-readable storage medium based on artificial intelligence, which can improve the noise reduction effect and efficiency of noise reduction processing.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a method for speech noise reduction based on artificial intelligence, including:

[0007] Acquire speech signals;

[0008] Acquire constraint conditions corresponding to the speech signal;

[0009] A noise reduction model adapted to the constraint condition is called to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal.

[0010] The present application provides a speech noise reduction device based on artificial intelligence, including:

[0011] An acquisition module, used for acquiring a voice signal;

[0012] An acquisition module, used for acquiring constraint conditions corresponding to the speech signal;

[0013] The noise reduction module is used to call the noise reduction model adapted to the constraint condition to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal.

[0014] In the above scheme, the acquisition module is also used to: before acquiring the voice signal, acquire the constraint conditions corresponding to the voice signal, and the constraint conditions include at least one of the planned time and planned location of the voice communication; the noise reduction module is also used to: before acquiring the voice signal, pre-acquire a noise reduction model adapted to the constraint conditions; when the constraint conditions are met, automatically call the noise reduction model adapted to the constraint conditions to perform noise reduction processing on the acquired voice signal to obtain a noise-reduced voice signal.

[0015] In the above scheme, the acquisition module is also used to acquire multiple noise reduction models before calling the noise reduction model adapted to the constraint condition to perform noise reduction processing on the speech signal, and the multiple noise reduction models are a collection of all noise reduction models that can be called, or a collection of noise reduction models adapted to the constraint condition; present options of the multiple noise reduction models in the human-computer interaction interface; in response to the selection operation, the selected noise reduction model is used as the noise reduction model adapted to the constraint condition.

[0016] In the above scheme, the acquisition module is also used to: acquire at least one of the following attribute information of the voice signal: time information of sending or receiving the voice signal, geographical location information of sending or receiving the voice signal, user information of sending or receiving the voice signal, and environment information when sending or receiving the voice signal; determine the constraint condition corresponding to the attribute information of the voice signal according to the preset correspondence between different attribute information and different constraint conditions.

[0017] In the above scheme, the noise reduction module is also used to: extract speech features from the speech signal; perform a first fully connected processing on the speech features corresponding to the speech signal through the first fully connected layer of the speech endpoint network to obtain a first fully connected processing result corresponding to the speech signal; perform speech endpoint detection processing on the first fully connected result corresponding to the speech signal through the first gated loop unit of the speech endpoint network to obtain a speech endpoint detection result corresponding to the speech signal; using the speech features corresponding to the speech signal, the first fully connected processing result corresponding to the speech signal and the speech endpoint detection result corresponding to the speech signal as input, predict the noise spectrum characteristics of the speech signal through the second gated loop unit of the noise spectrum prediction network; using the noise spectrum characteristics of the speech signal, the speech endpoint detection result corresponding to the speech signal and the speech features corresponding to the speech signal as input, predict the gain corresponding to the speech signal through the third gated loop unit of the noise spectrum removal network, and apply the gain corresponding to the speech signal to the speech signal to obtain a denoised speech signal.

[0018] In the above scheme, the noise reduction module is also used to: call the noise reduction model adapted to the constraint condition to perform noise reduction processing on the speech signal through the noise reduction model, and after obtaining the noise-reduced speech signal, determine the noise reduction effect parameters of the noise-reduced speech signal; when the noise reduction effect parameter is lower than the noise reduction effect parameter threshold, perform the following processing: determine the similarity between multiple other constraint conditions and the constraint conditions corresponding to the speech signal, and determine other constraint conditions with the highest similarity, so as to call the corresponding noise reduction model to perform noise reduction processing on the noise-reduced speech signal, and obtain an updated noise-reduced speech signal; wherein the other constraint conditions are different from the constraint conditions corresponding to the speech signal.

[0019] In the above scheme, each of the noise reduction models is adapted to a constraint condition, and different noise reduction models are adapted to different constraints; the device also includes a training module, which is used to obtain a set of training speech signal samples corresponding to multiple constraints one by one before obtaining the constraints corresponding to the speech signal; based on the set of training speech signal samples corresponding to multiple constraints one by one, multiple noise reduction models corresponding to multiple constraints are trained.

[0020] In the above scheme, each of the noise reduction models is adapted to at least one of the constraints, and different noise reduction models are adapted to different constraints; the training module is also used to obtain a set of training speech signal samples corresponding to multiple constraints one by one before obtaining the constraints corresponding to the speech signal; based on the set of training speech signal samples corresponding to multiple constraints, multiple noise reduction models corresponding to multiple constraints are trained; the multiple noise reduction models are clustered to obtain at least one clustered noise reduction model; wherein each clustered noise reduction model corresponds to at least one constraint.

[0021] In the above scheme, the training module is also used to obtain a set of test speech signal samples corresponding to each of the constraint conditions; the following processing is performed for each of the test speech signal sample sets: the test speech signal samples in the test speech signal sample set are subjected to denoising processing by using multiple denoising models to obtain denoising results corresponding one-to-one to the multiple denoising models; based on the denoising results obtained for each of the test speech signal sample sets and corresponding one-to-one to the multiple denoising models, the multiple denoising models are clustered to obtain at least one clustered denoising model.

[0022] In the above scheme, the training module is also used to perform the following processing for each of the test speech signal sample sets: determine multiple denoising results obtained after the test speech signal samples in the test speech signal sample set are denoised by multiple denoising models; determine the minimum mean square error of each of the denoising results, wherein the denoising result includes the denoised speech signal samples of multiple test speech signal samples in the test speech signal sample set; based on the minimum mean square error of the denoising results, sort the multiple denoising models in ascending order, and use at least one denoising model ranked first as at least one candidate denoising model corresponding to the test speech signal sample set; extract an intersection denoising model from the candidate denoising models corresponding to each of the test speech signal sample sets, so as to use the intersection denoising model as the clustered denoising model; wherein the candidate denoising model serving as the intersection denoising model corresponds to multiple test speech signal sample sets.

[0023] In the above scheme, the training module is also used to cluster the multiple constraints as original constraints to obtain at least one cluster constraint; obtain a test speech signal sample set corresponding to the multiple original constraints one by one, and a noise reduction model corresponding to the multiple original constraints one by one; fuse the test speech signal sample set corresponding to the multiple original constraints one by one to obtain a test speech signal sample set corresponding to the cluster constraint; perform noise reduction processing on the test speech signal samples in the test speech signal sample set corresponding to the cluster constraint through the multiple noise reduction models to obtain noise reduction results corresponding to the multiple noise reduction models; obtain the minimum mean square error of each noise reduction result, and determine the noise reduction model corresponding to the smallest minimum mean square error as the noise reduction model of the cluster.

[0024] In the above scheme, the training module is also used to obtain multiple noises carrying multiple attribute information; wherein the attribute information includes: time information of sending or receiving the voice signal, geographical location information of sending or receiving the voice signal, user information of sending or receiving the voice signal, and environmental information when sending or receiving the voice signal; according to the multiple attribute information, the multiple noises are divided and processed to obtain a noise set corresponding to the multiple constraint conditions one by one; wherein each constraint condition has the same multiple attribute information; the noise of each noise set is superimposed with the pure voice signal sample to obtain a training voice signal sample set corresponding to each constraint condition.

[0025] In the above scheme, the training module is also used to obtain the weight of the clean speech signal sample and the weight of the noise; according to the weight of the clean speech signal sample and the weight of the noise, the clean speech signal sample and the noise are weighted to obtain a training speech signal sample; based on the training speech signal sample, the clean speech signal or the noise is added to obtain a new training speech signal sample; and according to the training speech signal sample and the new training speech signal sample, the training speech signal sample set is formed.

[0026] In the above scheme, the training module is also used to perform denoising processing on the training speech signal samples included in the training speech signal sample set through the denoising model to obtain a denoised speech signal corresponding to the training speech signal sample; determine the error between the denoised speech signal corresponding to the training speech signal sample and the pure speech signal sample, and substitute the error into the loss function of the denoising model; determine the parameter change value of the denoising model when the loss function obtains the minimum value based on the learning rate of the denoising model, and update the parameters of the denoising model based on the parameter change value.

[0027] In the above scheme, the training module is also used to extract speech features from training speech signal samples; perform a first fully-connected processing on the speech features through the first fully-connected layer of the speech endpoint network to obtain a first fully-connected processing result; perform speech endpoint detection processing on the first fully-connected result through the first gated loop unit of the speech endpoint network to obtain a speech endpoint detection result; use the speech features, the first fully-connected processing result and the speech endpoint detection result as inputs, and predict the noise spectrum characteristics of the training speech signal samples through the second gated loop unit of the noise spectrum prediction network; use the noise spectrum characteristics, the speech endpoint detection result and the speech features as inputs, and predict the gain corresponding to the training speech signal samples through the third gated loop unit of the noise spectrum removal network, and apply the gain to the training speech signal samples.

[0028] The present application embodiment provides a method for training a noise reduction model based on artificial intelligence, including:

[0029] Acquire a plurality of training speech signal samples carrying noise in a plurality of constraint conditions to form a set of training speech signal samples corresponding one-to-one to the plurality of constraint conditions;

[0030] By using the noise reduction models corresponding to the multiple constraints one by one, performing noise reduction processing on the training speech signal samples included in the training speech signal sample set corresponding to the multiple constraints, so as to obtain a noise-reduced speech signal;

[0031] Determining an error between the denoised speech signal and a clean speech signal sample corresponding to the training speech signal sample, and updating parameters of the denoising model corresponding to the constraint condition according to the error;

[0032] The noise reduction model is used to perform noise reduction processing on the speech signal in the corresponding constraint conditions.

[0033] The present application embodiment provides a training device for a noise reduction model based on artificial intelligence, including:

[0034] A training module, used to obtain a plurality of training speech signal samples carrying noise in a plurality of constraint conditions, so as to form a set of training speech signal samples corresponding one by one to the plurality of constraint conditions;

[0035] The training module is further configured to perform noise reduction processing on the training speech signal samples included in the training speech signal sample set corresponding to the multiple constraints through noise reduction models corresponding to the multiple constraints one by one, so as to obtain noise-reduced speech signals;

[0036] The training module is further used to determine the error between the denoised speech signal and the clean speech signal sample corresponding to the training speech signal sample, and update the parameters of the denoising model corresponding to the constraint condition according to the error;

[0037] The noise reduction model is used to perform noise reduction processing on the speech signal in the corresponding constraint conditions.

[0038] An embodiment of the present application provides an electronic device, including:

[0039] A memory for storing executable instructions;

[0040] The processor is used to implement the artificial intelligence-based speech noise reduction method or the artificial intelligence-based noise reduction model training method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0041] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for implementing, when executed by a processor, an artificial intelligence-based speech noise reduction method or an artificial intelligence-based noise reduction model training method provided in an embodiment of the present application.

[0042] The embodiments of the present application have the following beneficial effects:

[0043] Different constraints of the applied speech signal are identified through the attribute information of the speech signal when it is applied, and the corresponding noise reduction model is called to perform noise reduction on the speech signal according to different constraints. That is, different suppression measures are taken according to different constraints, so as to optimize the noise reduction effect and improve the noise reduction efficiency in a targeted manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural diagram of a speech noise reduction system based on artificial intelligence provided in an embodiment of the present application;

[0045] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0046] Figures 3A-3D It is a flowchart of a speech noise reduction method based on artificial intelligence provided in an embodiment of the present application;

[0047] Figure 4 It is a schematic diagram of a noise reduction model of a speech noise reduction method based on artificial intelligence provided in an embodiment of the present application;

[0048] Figure 5A-5B Schematic diagram of constraint conditions of the artificial intelligence-based speech noise reduction method provided in an embodiment of the present application;

[0049] Fig. 6A It is a flowchart of a speech noise reduction method based on artificial intelligence provided in an embodiment of the present application;

[0050] Figure 6B It is a flowchart of the training method of the artificial intelligence-based denoising model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0052] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0053] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0055] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0056] 1) Noise spectrum characteristics: Frequency is one of the main parameters that describe sound characteristics and is a necessary condition for studying the distribution of sound intensity with frequency. The sound spectrum refers to the graph of the distribution of the intensity of the polyphonic sound (sound waves synthesized by simple harmonic components of different frequencies) with frequency. The sound spectrum composed of noise is the noise spectrum. Analyzing and understanding the components and properties of noise from the noise spectrum is called spectrum analysis. When performing spectrum analysis, it is usually necessary to understand whether the peak noise is in the low frequency, medium frequency or high frequency range to provide a basis for noise control.

[0057] The related technologies include ordinary noise reduction algorithms, which mainly include: adaptive filters, spectral subtraction, and Wiener filtering. This type of ordinary noise reduction algorithm usually assumes that the noise is additive random stationary noise, while speech has short-term stationary characteristics. Various statistical methods can be used to estimate the noise spectrum characteristics, and then the noisy speech is suppressed according to the calculated signal-to-noise ratio result. In the enhanced speech signal after noise suppression, the proportion of noise components is reduced, the speech signal-to-noise ratio is improved, and the speech is clearer and more understandable. However, the ordinary noise reduction algorithm is based on the assumption that the noise has additive random stationary characteristics. However, under actual constraints, the noise is divided into stationary noise and non-stationary noise. For non-stationary noise, some noise reduction methods in the related technologies cannot effectively suppress it and have obvious shortcomings.

[0058] The relevant technology includes AI noise reduction algorithm. The process of AI noise reduction algorithm is divided into deep learning training stage and deep learning reasoning stage. In the training stage, relevant speech time-frequency domain feature data, such as power spectrum, pitch period, speech endpoints and other feature data, are first extracted from a large number of speech and noise samples. These large amounts of data are used to train the designed deep network model (usually a multi-layer deep neural network, convolutional neural network, gated recurrent unit, long short-term memory network and other network structures). The purpose of training is to continuously optimize all parameters in the model. The result and goal of the optimization is to enable the deep network model to more accurately predict the pure speech components (or noise components) under various noisy speech, that is, to accurately identify noise and speech. AI noise reduction algorithm technology is a data-driven technology, that is, if the noise type of the actual application scenario has been covered in the sample library in the training stage, the AI ​​noise reduction algorithm can usually effectively suppress it and obtain cleaner speech. If the noise type of the actual scenario is not included in the training process, the AI ​​noise reduction algorithm may fail. Another disadvantage of the AI ​​noise reduction algorithm is that the effective speech under some constraints is close to individual noise, so the AI ​​algorithm noise reduction will misjudge and suppress the effective speech signal.

[0059] The embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for speech noise reduction based on artificial intelligence, which can adopt different suppression means for different constraints, optimize the noise reduction effect in a targeted manner and improve the noise reduction efficiency. The exemplary application of the electronic device provided by the embodiments of the present application is described below. The device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptops, tablet computers, desktop computers, set-top boxes, mobile devices (for example, mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), and can also be implemented as servers. Below, an exemplary application when the device is implemented as a server will be described.

[0060] See also Figure 1 , Figure 1It is a structural diagram of the voice noise reduction system based on artificial intelligence provided by the embodiment of the present application. In order to support a voice call application, the terminal 400-1 and the terminal 400-2 are connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two. The terminal 400-1 initiates a voice call request to the terminal 400-2 through the server 200. The voice call request carries a voice message. After the server sends the voice message to the terminal 400-2, the terminal 400-2 sends the attribute information of the voice message (the geographical information of the terminal 400-2 and the time information of receiving the voice message) to the server 200, so that the server 200 obtains the corresponding constraint conditions and calls the corresponding noise reduction model to perform noise reduction processing on the voice message, obtains the noise-reduced voice signal, and returns the noise-reduced voice signal to the terminal 400-2 for playback.

[0061] In some embodiments, an artificial intelligence-based voice noise reduction system is applied to a voice call application, and the voice call application includes a real-time voice call application and a non-real-time voice call application. The real-time voice call includes: the process of making a phone call and the process of conducting a voice call. The non-real-time voice call includes: the process of sending a voice message, a voice message, etc. The terminal 400-1 used by the user can receive voice information and send voice information. For the sent voice information, the noise reduction model running in the server 200 that is adapted to the constraint conditions of the sent voice information can be called to perform noise reduction processing on the sent voice information, and the noise-reduced voice signal is sent to the terminal 400-2 used by another user. For the received voice information, the terminal 400-1 can call the noise reduction model running in the server 200 that is adapted to the constraint conditions of the received voice information to perform noise reduction processing on the received voice information, and play the noise-reduced voice signal in the terminal 400-1.

[0062] In some embodiments, an artificial intelligence-based speech noise reduction system is applied to an application with a voice interaction function. The terminal 400-1 used by the user receives the user's voice information, which carries the content of the control instruction. For the user's voice information, a noise reduction model running in the server 200 that is adapted to the constraint conditions of the emitted voice information can be called to perform noise reduction processing on the emitted voice information, perform speech recognition processing on the noise-reduced voice signal, identify the control instruction, and return the control instruction to the terminal 400-1, so that the terminal 400-1 continues to respond to the identified control instruction.

[0063] In some embodiments, the electronic device can be implemented as a terminal. The terminal 400-1 used by the user receives voice information from the terminal 400-2 of other users. The terminal 400-1 directly calls the adapted local noise reduction model to perform noise reduction on the voice information according to the constraints of the received voice information (geographic location information and time information), and plays the noise reduction voice signal corresponding to the voice information on the terminal 400-1.

[0064] In some embodiments, server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminals 400-1 and 400-2 may be smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, etc., but are not limited thereto. The terminals and servers may be directly or indirectly connected via wired or wireless communications, which is not limited in the embodiments of the present invention.

[0065] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2 The server 200 shown includes: at least one processor 210, a memory 250, at least one network interface 220 and a user interface 230. The various components in the terminal 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 2 Various buses are labeled as bus system 240 .

[0066] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0067] The user interface 230 includes one or more output devices 231 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0068] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices that are physically remote from the processor 210.

[0069] The memory 250 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0070] In some embodiments, memory 250 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0071] Operating system 251, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0072] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220. Exemplary network interfaces 220 include: Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB).

[0073] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The figure shows an artificial intelligence-based speech noise reduction device 255-1 stored in the memory 250, which can be software in the form of a program and a plug-in, etc., including the following software modules: an acquisition module 2551, an acquisition module 2552, a noise reduction module 2553 and a training module 2554. Figure 2 A training device 255-2 for an artificial intelligence-based denoising model stored in a memory 250 is shown, which may be software in the form of programs and plug-ins, etc., and includes the following software modules: a training module 2555. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0074] The speech noise reduction method based on artificial intelligence provided by the embodiment of the present application will be described in combination with the exemplary application and implementation of the electronic device provided by the embodiment of the present application. The speech noise reduction method based on artificial intelligence provided by the embodiment of the present application can be implemented by a terminal or a server. For the received voice information, the terminal directly collects the voice signal from the voice information, and calls the noise reduction model adapted to the constraint conditions of the received voice information for noise reduction processing, or the terminal calls the noise reduction model adapted to the constraint conditions of the received voice information from the server for noise reduction processing.

[0075] See also Figure 3A , Figure 3A This is a flow chart of the speech noise reduction method based on artificial intelligence provided by the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.

[0076] In step 101, a speech signal is acquired.

[0077] As an example, the voice signal may be a voice signal collected in real time during a real-time voice call, where the real-time voice call includes: the process of making a phone call, the process of conducting a voice call. The voice signal may be a voice signal collected during a non-real-time voice call, where the non-real-time voice call includes: the process of sending a voice message, leaving a voice message, etc. The voice signal may also be collected from a human-computer interaction voice command issued by a user to a human-computer interaction interface.

[0078] In step 102, the constraint conditions corresponding to the speech signal are obtained.

[0079] In some embodiments, obtaining the constraint conditions corresponding to the voice signal in step 102 can be achieved by the following technical solution: obtaining at least one of the following attribute information of the voice signal: time information of sending or receiving the voice signal, geographical location information of sending or receiving the voice signal, user information of sending or receiving the voice signal, such as the person initiating the call, or the person receiving the call, and environmental information when sending or receiving the voice signal, such as a computer accepting a voice call or a mobile device accepting or initiating a voice call; determining the constraint conditions corresponding to the attribute information of the voice signal according to the preset correspondence between different attribute information and different constraint conditions.

[0080] As an example, attribute information is information used to characterize constraints related to voice signals. Attribute information includes one or more of time information, geographic location information, user information or environmental information. The constraints of voice signals include constraints for generating voice signals, constraints for sending voice signals or constraints for receiving voice signals. Therefore, time information includes time information for sending or receiving voice signals, geographic location information includes geographic location information for sending or receiving voice signals, user information includes user information for sending or receiving voice signals, such as the person who initiates the call or the person who receives the call, and environmental information includes environmental information in which the voice signal is sent or received, such as receiving or initiating a voice call through a computer or receiving or initiating a voice call through a mobile device. Computers and mobile devices characterize different environmental information. Computers characterize relatively fixed environmental information, such as an office environment, and mobile devices characterize changing environmental information, such as a leisure environment, etc. Any combination of several types of attribute information can define different constraints. There is a corresponding relationship between a single attribute information or a combination of attribute information and the constraints. For example, an airport from 10 a.m. to 11 a.m. is used as a constraint. Finally, based on the preset correspondence between different attribute information and different constraints, the constraint corresponding to the attribute information of the voice signal is determined.

[0081] In step 103, a noise reduction model adapted to the constraint condition is called to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal.

[0082] See also Figure 3B , Figure 3B is a flow chart of the speech noise reduction method based on artificial intelligence provided by the embodiment of the present application. In step 103, the noise reduction model adapted to the constraint condition is called to perform noise reduction processing on the speech signal. The noise reduction speech signal can be obtained by steps 1031-1035. Figure 3B The steps shown are explained.

[0083] In step 1031, speech features are extracted from the speech signal.

[0084] In step 1032, a first fully connected layer of the voice endpoint network performs a first fully connected processing on the voice feature to obtain a first fully connected processing result.

[0085] In step 1033, voice endpoint detection processing is performed on the first fully connected result through the first gated loop unit of the voice endpoint network to obtain a voice endpoint detection result.

[0086] In step 1034, the speech feature, the first fully connected processing result and the speech endpoint detection result are used as input, and the noise spectrum characteristics of the speech signal are predicted through the second gated recurrent unit of the noise spectrum prediction network.

[0087] In step 1035, the noise spectrum characteristics, speech endpoint detection results and speech features are used as input, the gain of the corresponding speech signal is predicted by the third gated recurrent unit of the noise spectrum removal network, and the gain is applied to the speech signal to obtain a noise-reduced speech signal.

[0088] As an example, see Figure 4 , Figure 4 It is a schematic diagram of a noise reduction model of a speech noise reduction method based on artificial intelligence provided in an embodiment of the present application. The noise reduction model includes a speech endpoint network, a noise spectrum estimation network, and a noise spectrum removal network. Speech features are extracted from speech signals. Speech features are extracted from time-frequency domain data, such as power spectrum, pitch period, speech endpoints, and other data. The speech features are subjected to a first fully connected processing through a first fully connected layer (24 dimensions) of the speech endpoint network. The first fully connected processing is implemented through a dense function and a tanh activation function to obtain a first fully connected processing result. The first fully connected result is subjected to speech endpoint detection processing through a first gated recurrent unit (24 dimensions) of the speech endpoint network to obtain a speech endpoint detection result. The gated recurrent unit is a commonly used gated recurrent neural network. The gated recurrent neural network receives the current input and the hidden state passed down from the previous node. This The hidden state contains the relevant information of the previous node. Combined with the hidden state and the current input, the gated recurrent neural network will obtain the output of the current hidden node and the hidden state passed to the next node. Through the iterative processing of multiple gated recurrent units and the mapping processing of the activation function, the speech endpoint detection result is obtained. The speech features, the first fully connected processing results and the speech endpoint detection results are used as inputs. The noise spectrum characteristics of the speech signal are predicted through the second gated recurrent unit (48 dimensions) of the noise spectrum prediction network. The noise spectrum characteristics, the speech endpoint detection results and the speech features are used as inputs. The third gated recurrent unit (96 dimensions) of the noise spectrum removal network predicts the gain (22 dimensions) of the corresponding speech signal, and the gain is applied to the speech signal to obtain a denoised speech signal. The data involved in steps 1031-1035 are all corresponding processing results obtained for the corresponding speech signal.

[0089] It should be noted that the execution order of the above steps 101, 102 and 103 is diverse, which is explained below.

[0090] In some embodiments, step 101 may be performed first, and then steps 102 and 103 may be performed in sequence. A specific example is described below. In step 101, a speech signal is obtained, and in step 102, the speech signal obtained in step 101 is analyzed to obtain the constraint conditions corresponding to the speech signal. In step 103, a noise reduction model adapted to the constraint conditions is called to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal.

[0091] When steps 101 to 103 are executed sequentially, step 102 is implemented as follows: obtaining at least one of the following attribute information of the voice signal: time information of sending or receiving the voice signal, geographical location information of sending or receiving the voice signal, user information of sending or receiving the voice signal, such as the person initiating the call, or the person receiving the call, and environmental information when sending or receiving the voice signal, such as a computer accepting a voice call or a mobile device accepting or initiating a voice call; determining the constraint conditions corresponding to the attribute information of the voice signal according to a preset correspondence between different attribute information and different constraint conditions.

[0092] As an example, the terminal receives a voice message, collects the voice information, responds to the user's play operation on the voice information, obtains the play time information and the play location information of the voice information, obtains the constraints corresponding to the play time information and the play location information, and calls the noise reduction model corresponding to the constraints to perform noise reduction on the voice signal, obtains the noise reduced voice signal, and plays the noise reduced voice signal.

[0093] In some embodiments, step 102 may be performed first, and then step 103 and step 101 may be performed. An example is described below. In step 102, that is, before acquiring the voice signal, the constraint conditions corresponding to the voice signal are acquired, and the constraint conditions include at least one of the planned time and planned location of the voice communication. In step 103, a noise reduction model adapted to the constraint conditions is acquired in advance, and then the voice signal is acquired by proceeding to step 101. When the constraint conditions are met, the voice signal is acquired by continuing to proceed to step 103 to automatically call the noise reduction model adapted to the constraint conditions to perform noise reduction processing on the acquired voice signal to obtain a noise-reduced voice signal.

[0094] As an example, for the voice signal in a remote conference, the planned time and location of the remote conference are extracted from the schedule function and / or the to-do function of the user's instant messaging client, or the planned time and location of the remote conference are extracted from the notes of the voice assistant. The planned time and location of the remote conference constitute the constraints of the voice signal transmitted in the remote conference. The planned locations include airports, offices, etc. Before starting the remote conference, a noise reduction model adapted to the constraints is obtained in advance. For example, a noise reduction model adapted to the constraints is obtained in advance from a server, or a noise reduction model adapted to the constraints is pre-loaded on the processing end (terminal or server). When the constraints are met, that is, when the user arrives at the planned meeting location at the planned meeting time, the noise reduction model adapted to the constraints is automatically called to perform noise reduction processing on the acquired voice signal to obtain a noise-reduced voice signal, thereby saving time for real-time adaptation and calling of the noise reduction model and improving noise reduction efficiency.

[0095] As an example, for a voice call scenario, the planned time of the voice call is extracted from the voice assistant's memo (the planned location of the voice call is usually not recorded in the memo), and the geographical location information of the terminal is obtained when the voice call is broadcast but not connected. The geographical location information and the planned time constitute the constraints of the voice signal to be transmitted in the voice call. Before the voice call is connected, a noise reduction model adapted to the constraints is pre-acquired, for example, a noise reduction model adapted to the constraints is pre-acquired from the server, or a noise reduction model adapted to the constraints is pre-loaded on the processing end (terminal or server). After the voice call is connected, when the constraints are met (the time point when the constraints are met may be before the voice call is connected), the noise reduction model adapted to the constraints is automatically called to perform noise reduction processing on the acquired voice signal to obtain a noise-reduced voice signal, thereby saving time for real-time adaptation and calling of the noise reduction model and improving noise reduction efficiency.

[0096] In some embodiments, step 101 and step 102 may be performed first, and then the noise reduction model selected by the user may be obtained according to the constraint conditions obtained in step 102, and then step 103 may be performed according to the noise reduction model selected by the user. An example is described below. In step 101, a speech signal is obtained, and in step 102, the speech signal obtained in step 101 is analyzed to obtain the constraint conditions corresponding to the speech signal. Before calling the noise reduction model adapted to the constraint conditions in step 103 to perform noise reduction processing on the speech signal, multiple noise reduction models are obtained, and the multiple noise reduction models are a collection of all noise reduction models that can be called, or a collection of noise reduction models adapted to the constraint conditions (obtaining the constraint conditions through step 102); multiple noise reduction model options are presented in the human-computer interaction interface; in response to the selection operation, the selected noise reduction model is used as the noise reduction model adapted to the constraint conditions, and then step 103 is performed according to the noise reduction model selected by the user.

[0097] As an example, for a remote conference scenario, after step 102 and before step 103, multiple noise reduction model options are presented in the human-computer interaction interface, and the options correspond to different noise reduction models. The noise reduction models corresponding to the presented options are all noise reduction models, or noise reduction models that are adapted to the constraints. If the noise reduction models corresponding to the presented options are all noise reduction models, the options corresponding to the noise reduction models that are adapted to the constraints are highlighted, and in response to the user's selection operation, the selected noise reduction model is updated to a noise reduction model that is adapted to the constraints. In the remote conference scenario, the user's selection operation can occur during the remote conference.

[0098] As an example, in a remote conference scenario, the user's selection operation can occur before the remote conference, and the planned time and planned location of the remote conference are extracted from the voice assistant's notes. The planned time and planned location of the remote conference constitute the constraints of the voice signal transmitted in the remote conference. Before starting the remote conference, a noise reduction model that is adapted to the constraints is pre-acquired, for example, a noise reduction model that is adapted to the constraints is pre-acquired from the server, or a noise reduction model that is adapted to the constraints is pre-loaded on the processing end (terminal or server), and multiple noise reduction model options are presented in the human-computer interaction interface, and the options correspond to different noise reduction models, and the options presented are The noise reduction model corresponding to the item is all noise reduction models, or is a pre-acquired noise reduction model adapted to the constraints. If the noise reduction model corresponding to the presented option is all noise reduction models, the option corresponding to the pre-acquired noise reduction model adapted to the constraints is highlighted, and in response to the user selection operation, the selected noise reduction model is updated to the noise reduction model adapted to the constraints. After the meeting starts, voice signals are collected, and the noise reduction model adapted to the constraints is automatically called to perform noise reduction processing on the acquired voice signals to obtain noise-reduced voice signals, thereby accurately meeting user needs and improving the targeted noise reduction while improving the user's human-computer interaction efficiency.

[0099] See also Figure 3C , Figure 3C is a flow chart of the speech denoising method based on artificial intelligence provided by an embodiment of the present application. In step 103, a denoising model adapted to the constraint condition is called to denoise the speech signal through the denoising model. After obtaining the denoised speech signal, steps 104-105 can be executed to combine Figure 3C The steps shown are explained.

[0100] In step 104, the noise reduction effect parameters of the noise reduction speech signal are determined.

[0101] In step 105, when the noise reduction effect parameter is lower than the noise reduction effect parameter threshold, the similarity between multiple other constraints and the constraints of the corresponding speech signal is determined, and the other constraints with the highest similarity are determined to call the corresponding noise reduction model to perform noise reduction processing on the noise reduction speech signal to obtain an updated noise reduction speech signal.

[0102] As an example, the noise reduction effect parameters of the noise reduction speech signal are determined. The noise reduction effect parameters are parameters obtained based on the signal-to-noise ratio of the noise reduction speech signal, the noise spectrum intensity, etc. The noise reduction effect threshold is a threshold parameter calculated based on multiple set parameters (signal-to-noise ratio, noise spectrum intensity, etc.). The noise reduction effect parameter is lower than the noise reduction effect parameter threshold, which indicates that the noise reduction effect is not ideal and noise reduction processing is required. Therefore, the similarity between multiple other constraints and the constraints of the corresponding speech signal is determined, and the other constraints with the highest similarity are determined, so as to call the noise reduction model corresponding to the other constraints to perform noise reduction processing on the noise reduction speech signal, and obtain the noise reduced speech signal. The other constraints are different from the constraints of the corresponding speech signal. Through noise reduction processing, the noise reduction quality and noise reduction efficiency can be effectively improved.

[0103] In some embodiments, each noise reduction model is adapted to a constraint condition, and different noise reduction models are adapted to different constraints; before obtaining the constraints corresponding to the speech signal in step 102, the following technical solution can be executed: obtaining a set of training speech signal samples corresponding one-to-one to multiple constraints; based on the set of training speech signal samples corresponding one-to-one to multiple constraints, training to obtain multiple original noise reduction models corresponding one-to-one to multiple constraints.

[0104] As an example, multiple training speech signal samples carrying noise in multiple constraints are obtained to form a set of training speech signal samples corresponding one-to-one to the multiple constraints, and noise reduction models corresponding one-to-one to the multiple constraints are initialized; the training speech signal samples included in the set of training speech signal samples corresponding to the multiple constraints are subjected to noise reduction processing by means of the noise reduction models corresponding one-to-one to the multiple constraints to obtain noise-reduced speech signals; the error between the noise-reduced speech signal and the pure speech signal sample corresponding to the training speech signal sample is determined, and the parameters of the noise reduction model corresponding to the constraints are updated according to the error, wherein the noise reduction model is used to perform noise reduction processing on the speech signal in the corresponding constraints.

[0105] In some embodiments, each noise reduction model is adapted to at least one constraint condition, and different noise reduction models are adapted to different constraints; before obtaining the constraints corresponding to the speech signal, the following technical scheme can be executed: obtaining a set of training speech signal samples corresponding one-to-one to multiple constraints; based on the set of training speech signal samples corresponding one-to-one to multiple constraints, training to obtain multiple original noise reduction models corresponding one-to-one to multiple constraints; clustering the multiple original noise reduction models to obtain at least one clustered noise reduction model; wherein each clustered noise reduction model corresponds to at least one constraint condition.

[0106] As an example, a noise reduction model corresponding to each constraint condition obtained through training based on multiple constraint conditions is used as the original noise reduction model, and the original noise reduction model is clustered to obtain at least one clustered noise reduction model. After clustering, one noise reduction model can perform noise reduction corresponding to multiple constraint conditions. For example, the noise reduction model corresponding to the constraint condition of being at the airport from 8 am to 10 am can also be used to process speech signals with the constraint condition of being at the airport from 10 pm to 12 am. Clustering of noise reduction models can effectively reduce the occupancy of storage resources and improve the utilization rate of noise reduction models.

[0107] In some embodiments, the above-mentioned clustering processing of multiple noise reduction models to obtain at least one clustered noise reduction model can be achieved through the following technical solution: obtaining a test speech signal sample set corresponding to each constraint condition; performing the following processing on each test speech signal sample set: performing noise reduction processing on the test speech signal samples in the test speech signal sample set through multiple noise reduction models to obtain noise reduction results corresponding one-to-one to the multiple noise reduction models, that is, noise-reduced speech signal samples; clustering the multiple noise reduction models according to the noise reduction results obtained for each test speech signal sample set and corresponding one-to-one to the multiple noise reduction models to obtain at least one clustered noise reduction model.

[0108] As an example, take M groups of test speech signal sample sets as test sets, each test set has a trained noise reduction model (without clustering processing), and for a certain test set, use the trained M noise reduction models to perform noise reduction processing on the test speech signal samples of the test set to obtain M noise reduction results (corresponding to different noise reduction models respectively), and then according to the noise reduction results obtained for each test speech signal sample set and corresponding one-to-one to the multiple noise reduction models, cluster processing is performed on the multiple noise reduction models to obtain at least one clustered noise reduction model.

[0109] In some embodiments, the above-mentioned clustering processing of multiple noise reduction models based on the noise reduction results obtained for each test speech signal sample set and corresponding one-to-one to multiple noise reduction models to obtain at least one clustered noise reduction model can be achieved by the following technical scheme: performing the following processing for each test speech signal sample set: determining multiple noise reduction results obtained after the test speech signal samples in the test speech signal sample set are subjected to noise reduction processing by multiple noise reduction models; determining the minimum mean square error of each noise reduction result, wherein the noise reduction result includes the noise reduction speech signal samples of the multiple test speech signal samples in the test speech signal sample set; sorting the multiple noise reduction models in ascending order based on the minimum mean square error of the noise reduction results, and taking at least one noise reduction model ranked first as the candidate noise reduction model corresponding to the test speech signal sample set; extracting the intersection noise reduction model from the candidate noise reduction models corresponding to each test speech signal sample set, so as to take the intersection noise reduction model as the clustered noise reduction model; wherein the candidate noise reduction model as the intersection noise reduction model corresponds to multiple test speech signal sample sets.

[0110] As an example, following the above embodiment, M denoising results are sorted with the minimum mean square error as the evaluation criterion, and each denoising result includes a denoised test speech signal of a certain denoising model for all test speech signal samples in a certain test speech signal sample set. Assuming that the test speech signal sample set includes 10 test speech signal samples, 10 denoised test speech signals will be obtained, and the minimum mean square error of these 10 denoised test speech signals will be determined as the minimum mean square error of the denoising model for a specific test speech signal sample set. Based on M denoising models, M minimum mean square errors for a certain test speech signal sample set can be obtained, and the denoising models corresponding to the first K denoising results whose minimum mean square error values ​​are less than the error threshold are selected as candidate denoising models for the test speech signal sample set. If the number of denoising models corresponding to the denoising results whose minimum mean square error values ​​are less than the error threshold is less than K, the actual number shall prevail. If there is no denoising result whose minimum mean square error value is less than the error threshold, the denoising model corresponding to the denoising result with the smallest minimum mean square error value is selected. Model is used as the only candidate denoising model for the test speech signal sample set. If there are models with intersections in the candidate denoising models corresponding to the M test speech signal sample sets, for example, the test speech signal sample set A has two corresponding candidate denoising models (a, b), and the test speech signal sample set B has three corresponding candidate denoising models (a, b, c), then the candidate denoising model (a, b) is used as the intersection denoising model corresponding to the test speech signal sample set A and the test speech signal sample set B. These test speech signal sample sets with intersections uniformly use the model with the smallest comprehensive error in the intersection denoising model as the final clustering denoising model corresponding to the test speech signal sample set A and the test speech signal sample set B. When there is only one intersection denoising model for the test speech signal sample set A and the test speech signal sample set B, the intersection denoising model is directly used as the final clustering denoising model. The comprehensive error can be the minimum mean square error or other evaluation parameters. The comprehensive error is the comprehensive denoising evaluation result of multiple intersection denoising models for multiple test speech signal sample sets.

[0111] In some embodiments, the above-mentioned clustering processing of multiple original noise reduction models to obtain at least one clustered noise reduction model can be achieved through the following technical solutions: clustering multiple constraints as original constraints to obtain at least one clustered constraint; obtaining a test speech signal sample set corresponding to the multiple original constraints one by one, and an original noise reduction model corresponding to the multiple original constraints one by one; fusing the test speech signal sample set corresponding to the multiple original constraints one by one to obtain a test speech signal sample set corresponding to the clustering constraints; performing noise reduction processing on the test speech signal samples in the test speech signal sample set corresponding to the clustering constraints through multiple noise reduction models to obtain noise reduction results corresponding to the multiple noise reduction models one by one; obtaining the minimum mean square error of each noise reduction result, and determining the noise reduction model corresponding to the smallest minimum mean square error as the clustered noise reduction model.

[0112] As an example, different from the above embodiment, multiple constraints can be clustered as original constraints to obtain at least one cluster constraint; that is, the constraints are first clustered, and then the noise reduction model corresponding to each cluster constraint is obtained as the cluster noise reduction model. Assuming that there are M original constraints, corresponding M test speech signal sample sets and corresponding M original noise reduction models, N constraints are obtained after the constraints are clustered, and then N test speech signal sample sets are obtained. Through the M noise reduction models, the test speech signal samples in the test speech signal sample set of a cluster constraint are subjected to noise reduction processing to obtain M test speech signal sample sets. denoising results; the minimum mean square error of each denoising result is obtained, and the denoising model corresponding to the smallest minimum mean square error is determined as the denoising model of the clustering constraint (clustering denoising model), so as to obtain the denoising model of each clustering constraint in turn. The clustering denoising models corresponding to different clustering constraints may have an intersection, that is, the denoising model of clustering constraint A is a, and the denoising model of clustering constraint B is also a. At this time, the denoising model can be further reused, and the denoising model a will be automatically reused by the two clustering constraints. The clustering of the denoising model can effectively reduce the occupancy of storage resources and improve the utilization rate of the denoising model.

[0113] In some embodiments, the above-mentioned acquisition of a training speech signal sample set corresponding to multiple constraints can be achieved through the following technical solution: acquiring multiple noises carrying multiple attribute information; wherein the attribute information includes: time information of sending or receiving the speech signal, geographical location information of sending or receiving the speech signal, user information of sending or receiving the speech signal, and environmental information when sending or receiving the speech signal; dividing and processing the multiple noises according to the multiple attribute information to obtain a noise set corresponding to multiple constraints; wherein each constraint has the same multiple attribute information; superimposing the noise of each noise set with the pure speech signal sample to obtain a training speech signal sample set corresponding to each constraint.

[0114] In some embodiments, the above-mentioned superposition processing of the noise of each noise set with the clean speech signal sample to obtain the training speech signal sample set corresponding to each constraint condition can be achieved through the following technical scheme: obtaining the weight of the clean speech signal sample and the weight of the noise; according to the weight of the clean speech signal sample and the weight of the noise, weighted processing is performed on the clean speech signal sample and the noise to obtain the training speech signal sample; based on the training speech signal sample, the clean speech signal or noise is added to obtain a new training speech signal sample; according to the training speech signal sample and the new training speech signal sample, a training speech signal sample set is formed.

[0115] As an example, taking time information and geographic location information as an example, the time information and geographic location information form corresponding constraints in the form of two-dimensional information, and all noises are divided into M groups (M is an integer greater than or equal to 2). The collected noise is divided into M groups of different noises according to the time information and geographic location information. For example, the time of the first group of noises is 10:00-11:00 in the morning, and the geographic coordinates are GPS0. The time of the second group of noises is 11:00-12:00 in the morning, and the geographic coordinates are GPS0. And so on. Each group of noises and the pure speech signal are linearly superimposed with different weighted values ​​to obtain a certain scale of training speech signal sample set. For the training of the noise reduction model, after each noise reduction model is trained separately, the noise reduction model corresponding to each time period and geographical location (constraint conditions) is obtained. M groups of noise sample sets (each group corresponds to a different time and geographical location, that is, corresponds to a different constraint condition) are constructed to obtain M training speech signal sample sets. A larger scale of training speech signal samples can be generated by linear superposition of different parameters (for example, the weighted value of the pure speech signal is 0.7, and the weighted value of the noise is 0.3). As the scale of the training speech signal sample set increases, the deep network of the noise reduction model has better adaptability to the constraint conditions, and can better identify and suppress the noise under different constraint conditions.

[0116] In some embodiments, the above-mentioned training of multiple noise reduction models corresponding to multiple constraints based on a training speech signal sample set corresponding to multiple constraints can be achieved through the following technical solutions: denoising the training speech signal samples included in the training speech signal sample set through the noise reduction model to obtain a noise-reduced speech signal corresponding to the training speech signal sample; determining the error between the noise-reduced speech signal corresponding to the training speech signal sample and the pure speech signal sample, and substituting the error into the loss function of the noise reduction model; determining the parameter change value of the noise reduction model when the loss function obtains the minimum value based on the learning rate of the noise reduction model, and updating the parameters of the noise reduction model based on the parameter change value.

[0117] In some embodiments, the above-mentioned denoising processing of the training speech signal samples included in the training speech signal sample set by the denoising model can be achieved by the following technical scheme: extracting speech features from the training speech signal samples; performing a first fully-connected processing on the speech features through the first fully-connected layer of the speech endpoint network to obtain a first fully-connected processing result; performing a speech endpoint detection processing on the first fully-connected result through the first gated loop unit of the speech endpoint network to obtain a speech endpoint detection result; taking the speech features, the first fully-connected processing result and the speech endpoint detection result as input, predicting the noise spectrum characteristics of the training speech signal samples through the second gated loop unit of the noise spectrum prediction network; taking the noise spectrum characteristics, the speech endpoint detection result and the speech features as input, predicting the gain of the corresponding training speech signal samples through the third gated loop unit of the noise spectrum removal network, and applying the gain to the training speech signal samples.

[0118] As an example, a denoising model includes a speech endpoint network, a noise spectrum prediction network and a noise spectrum removal network. Speech features are extracted from training speech signal samples. The speech features are extracted from time-frequency domain data, such as power spectrum, pitch period, speech endpoints and the like. The speech features are subjected to a first fully-connected processing through a first fully-connected layer of the speech endpoint network to obtain a first fully-connected processing result corresponding to the training speech signal sample. The first fully-connected result corresponding to the training speech signal sample is subjected to speech endpoint detection processing through a first gated cyclic unit of the speech endpoint network to obtain a speech endpoint detection result corresponding to the training speech signal sample. The speech features corresponding to the training speech signal sample, the first fully-connected processing result corresponding to the training speech signal sample and the speech endpoint detection result corresponding to the training speech signal sample are used as input. The second gated recurrent unit of the spectrum estimation network predicts the noise spectrum characteristics of the training speech signal samples, and takes the noise spectrum characteristics of the corresponding training speech signal samples, the speech endpoint detection results of the corresponding training speech signal samples, and the speech features of the corresponding training speech signal samples as inputs, and predicts the gain of the corresponding training speech signal samples through the third gated recurrent unit of the noise spectrum removal network, and applies the gain of the corresponding training speech signal samples to the training speech signal samples to obtain the denoised speech signal of the corresponding training speech signal samples, determines the error between the denoised speech signal of the corresponding training speech signal samples and the pure speech signal samples, and substitutes the error into the loss function of the denoising model, determines the parameter change value of the denoising model when the loss function obtains the minimum value based on the learning rate of the denoising model, and updates the parameters of the denoising model based on the parameter change value.

[0119] See also Figure 3D , Figure 3D is a flow chart of a training method for a noise reduction model based on artificial intelligence provided in an embodiment of the present application, which will be combined with Figure 3D The steps shown are explained.

[0120] In step 201, a plurality of training speech signal samples carrying noise in a plurality of constraint conditions are obtained to form a set of training speech signal samples corresponding to the plurality of constraint conditions one by one, and a noise reduction model corresponding to the plurality of constraint conditions one by one is initialized;

[0121] In step 202, a noise reduction model corresponding to each of the plurality of constraints is used to perform noise reduction processing on the training speech signal samples included in the training speech signal sample set corresponding to the plurality of constraints to obtain a noise-reduced speech signal;

[0122] In step 203, the error between the denoised speech signal and the clean speech signal sample corresponding to the training speech signal sample is determined, and the parameters of the denoising model corresponding to the constraint condition are updated according to the error;

[0123] The noise reduction model is used to perform noise reduction processing on the speech signal in the corresponding constraint conditions.

[0124] The specific implementation of steps 201-203 can refer to the embodiments corresponding to steps 101-103.

[0125] The following is an explanation of an exemplary application of an embodiment of the present application in an actual application constraint condition.

[0126] Different sound samples collected through system time and geographic location are used to independently train multiple noise reduction models, and the trained noise reduction models are clustered to obtain noise reduction models with deep networks that match different constraints (time, geographic location), thereby effectively improving the noise reduction effect and efficiency.

[0127] See also Figure 5A-5B , Figure 5A-5B is a schematic diagram of constraint conditions of the speech noise reduction method based on artificial intelligence provided in an embodiment of the present application, Figure 5A A schematic diagram of roads and streets from 8 to 10 a.m. is shown, which has specific noise reduction requirements and noise characteristics under the constraints of roads and streets and corresponding time conditions. A schematic diagram of airports from 8 to 10 a.m. is shown, which has specific noise reduction requirements and noise characteristics under the constraints of airports and corresponding time conditions. A schematic diagram of offices from 7 to 9 p.m. is shown, which has specific noise reduction requirements and noise characteristics under the constraints of offices and corresponding time conditions. A schematic diagram of subway stations from 3 to 5 p.m. is shown, which has specific noise reduction requirements and noise characteristics under the constraints of subway stations and corresponding time conditions. Constraints are identified for the two-dimensional information composed of geographic location information and time information, and constraints 1-4 are identified respectively, and noise reduction models AD are called accordingly.

[0128] See also Fig. 6A , Fig. 6A is a flow chart of the speech noise reduction method based on artificial intelligence provided by the embodiment of the present application, see Figure 6B , Figure 6B It is a flow chart of the training method of the noise reduction model based on artificial intelligence provided in an embodiment of the present application. First, the noise reduction model is trained, and then the noise reduction model is called for reasoning. The training phase can be run on the server, and finally N types of noise reduction models are obtained (N is an integer greater than or equal to 2), and the reasoning phase can be run in real time on the terminal or on the server to obtain a noise-reduced speech signal.

[0129] In some embodiments, it is preferred to obtain a set of training speech signal samples for training. The training speech signal sample set needs to be labeled in advance, that is, the attributes of each frame signal need to be confirmed, such as whether it is noise or speech, and the training process needs to provide an expected output result (that is, pure speech after noise is suppressed in an ideal situation) as a guide for the denoising model to perform parameter optimization. Since the denoising model training requires a large number of labeled samples and expected output results, in order to meet the requirements of a trainable model network, the training speech signal sample set is synthesized in a constructed manner, that is, the pure speech signal is linearly superimposed with noise, so that a noisy speech signal is synthesized as a speech signal sample for training input, and the expected output signal is the pure speech signal before synthesis. By linearly superimposing different parameters (for example, the weighted value of the pure speech signal in a single frame signal is 0.7, and the weighted value of the noise is 0.3), a larger scale of training speech signal samples can be generated, which is sufficient to support the training process of the denoising model, and this sample construction method can well provide the expected data corresponding to each sample to guide the optimization training of the denoising model.

[0130] In some embodiments, system time information and geographic coordinate information can be read from the terminal, and corresponding constraints are formed according to this two-dimensional information, and all noises are divided into M groups (M is an integer greater than or equal to 2), that is, the collected noise set is divided into M groups of different samples according to time and coordinates. For example, the time of the first group of noise samples is 10:00-11:00 in the morning, and the geographic coordinates are GPS0, and the time of the second group of noise samples is 11:00-12:00 in the morning, and the geographic coordinates are GPS0, and so on. Each group of noise and the clean voice signal (which can be purchased or found on an open source website) are linearly superimposed with different weighted values ​​to obtain a certain regularity. The training speech signal sample set of the model is used to train the noise reduction model. After each noise reduction model is trained separately, the noise reduction model corresponding to each time period and geographical location (constraint condition) is obtained. For example: M groups of noise sample sets (each group corresponds to a different time and geographical location, that is, corresponds to different constraints), M training speech signal sample sets are constructed, and M noise reduction models are trained. The noise reduction models can be designed with the same structure or with different structures. With the increase in the scale of the training speech signal sample set and the increase in parameters, the deep network of the noise reduction model has better adaptability to constraints, and can better identify and suppress noise under different constraints.

[0131] In some embodiments, in order to simplify the number of models and reduce storage resources, the noise reduction model can be clustered to obtain N types of noise reduction models (M>=N), and the N noise reduction models correspond to the constraints obtained by combining M different time coordinates. In practical applications, for example, if the user makes a call at 11:00-12:00 in the morning and the geographic coordinates are GPS0, the server will call the noise reduction model corresponding to the second set of training voice signal sample sets to perform real-time noise reduction processing, see Figure 4 , Figure 4 is a schematic diagram of a noise reduction model of a speech noise reduction method based on artificial intelligence provided in an embodiment of the present application, through Figure 4 The noise reduction model shown performs noise reduction processing on the training speech signal samples, that is, accurately identifies the noise and speech signal, and then suppresses the noise through spectral subtraction or Wiener filtering post-processing to obtain a noise-reduced speech signal (clean speech signal). The inference stage is the noise reduction process of the actual call process, which uses the trained noise reduction model (the network unit parameters have been relatively optimal during the training process) to estimate the speech or noise of the collected noisy speech signal, and finally suppresses the noise component to obtain a cleaner speech signal.

[0132] In some embodiments, denoising model clustering: In order to streamline the number of models and reduce storage resource consumption, we propose a denoising model clustering method here to compress M denoising models into N denoising models (M>=N). The process is: first prepare M groups of noisy sample sets as denoising test sets, corresponding to M time and geographic coordinates respectively, and the number of samples does not need to be large; use the trained M denoising models to run each group of noisy test samples, that is, AI denoising processing, to obtain M denoising results, and use the minimum mean square error as the judgment criterion to sort the M results, and select the first K model results with error values ​​less than a preset error threshold as candidate denoising models for the group of test sample sets. If the number of models that meet the error condition is less than K, the actual number shall prevail. If the errors of all models are greater than the preset threshold value, the model with the smallest error is selected as the only candidate denoising model for the sample set; if there are models with intersections corresponding to the candidate denoising models of the M sample sets, these sample groups with intersections will uniformly use the model with the smallest comprehensive error in the intersection model as their final denoising model. This clustering method can reduce the number of models.

[0133] The following is a description of an exemplary structure of the artificial intelligence-based speech noise reduction device 255 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2As shown, the software modules stored in the artificial intelligence-based speech noise reduction device 255-1 in the memory 250 may include: an acquisition module 2551, used to acquire a speech signal; an acquisition module 2552, used to acquire the constraint conditions corresponding to the speech signal; and a noise reduction module 2553, used to call a noise reduction model adapted to the constraint conditions to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal.

[0134] In some embodiments, the acquisition module 2552 is also used to: before acquiring the voice signal, acquire the constraint conditions corresponding to the voice signal, the constraint conditions including at least one of the planned time and planned location of the voice communication; the noise reduction module is also used to: before acquiring the voice signal, pre-acquire a noise reduction model adapted to the constraint conditions; when the constraint conditions are met, automatically call the noise reduction model adapted to the constraint conditions to perform noise reduction processing on the acquired voice signal to obtain a noise-reduced voice signal.

[0135] In some embodiments, the acquisition module 2552 is also used to acquire multiple noise reduction models before calling the noise reduction model adapted to the constraint conditions to perform noise reduction processing on the speech signal. The multiple noise reduction models are a collection of all noise reduction models that can be called, or a collection of noise reduction models adapted to the constraint conditions; options for multiple noise reduction models are presented in the human-computer interaction interface; in response to a selection operation, the selected noise reduction model is used as the noise reduction model adapted to the constraint conditions.

[0136] In some embodiments, the acquisition module 2552 is also used to: obtain at least one of the following attribute information of the voice signal: time information of sending or receiving the voice signal, geographical location information of sending or receiving the voice signal, user information of sending or receiving the voice signal, and environmental information when sending or receiving the voice signal; determine the constraint conditions corresponding to the attribute information of the voice signal according to the preset correspondence between different attribute information and different constraint conditions.

[0137] In some embodiments, the noise reduction module 2553 is also used to: extract speech features from the speech signal; perform a first fully-connected processing on the speech features of the corresponding speech signal through the first fully-connected layer of the speech endpoint network to obtain a first fully-connected processing result of the corresponding speech signal; perform speech endpoint detection processing on the first fully-connected result of the corresponding speech signal through the first gated loop unit of the speech endpoint network to obtain a speech endpoint detection result of the corresponding speech signal; take the speech features of the corresponding speech signal, the first fully-connected processing result of the corresponding speech signal and the speech endpoint detection result of the corresponding speech signal as input, and predict the noise spectrum characteristics of the speech signal through the second gated loop unit of the noise spectrum prediction network; take the noise spectrum characteristics of the speech signal, the speech endpoint detection result of the corresponding speech signal and the speech features of the corresponding speech signal as input, and predict the gain of the corresponding speech signal through the third gated loop unit of the noise spectrum removal network, and apply the gain of the corresponding speech signal to the speech signal to obtain a denoised speech signal.

[0138] In some embodiments, the noise reduction module 2553 is also used to: call a noise reduction model that is adapted to the constraint conditions to perform noise reduction processing on the speech signal through the noise reduction model, and after obtaining the noise-reduced speech signal, determine the noise reduction effect parameters of the noise-reduced speech signal; when the noise reduction effect parameters are lower than the noise reduction effect parameter threshold, perform the following processing: determine the similarity between multiple other constraint conditions and the constraint conditions of the corresponding speech signal, and determine the other constraint conditions with the highest similarity, so as to call the corresponding noise reduction model to perform noise reduction processing on the noise-reduced speech signal, and obtain an updated noise-reduced speech signal; wherein the other constraint conditions are different from the constraint conditions of the corresponding speech signal.

[0139] In some embodiments, each noise reduction model is adapted to a constraint condition, and different noise reduction models are adapted to different constraints; the device also includes a training module 2554, which is used to obtain a set of training speech signal samples corresponding to multiple constraints one by one before obtaining the constraints corresponding to the speech signal; based on the set of training speech signal samples corresponding to multiple constraints one by one, multiple noise reduction models corresponding to multiple constraints are trained.

[0140] In some embodiments, each noise reduction model is adapted to at least one constraint condition, and different noise reduction models are adapted to different constraints; the training module 2554 is also used to obtain a set of training speech signal samples corresponding to multiple constraints one by one before obtaining the constraints corresponding to the speech signal; based on the set of training speech signal samples corresponding to multiple constraints one by one, train multiple noise reduction models corresponding to multiple constraints one by one; cluster the multiple noise reduction models to obtain at least one clustered noise reduction model; wherein each clustered noise reduction model corresponds to at least one constraint condition.

[0141] In some embodiments, the training module 2554 is also used to obtain a set of test speech signal samples corresponding to each constraint condition; perform the following processing on each test speech signal sample set: perform noise reduction processing on the test speech signal samples in the test speech signal sample set through multiple noise reduction models to obtain noise reduction results corresponding one-to-one to the multiple noise reduction models; based on the noise reduction results obtained for each test speech signal sample set and corresponding one-to-one to the multiple noise reduction models, cluster the multiple noise reduction models to obtain at least one clustered noise reduction model.

[0142] In some embodiments, the training module 2554 is also used to perform the following processing for each test speech signal sample set: determine multiple noise reduction results obtained after the test speech signal samples in the test speech signal sample set are subjected to noise reduction processing by multiple noise reduction models; determine the minimum mean square error of each noise reduction result, wherein the noise reduction result includes the noise reduction speech signal sample of multiple test speech signal samples in the test speech signal sample set; based on the minimum mean square error of the noise reduction result, sort the multiple noise reduction models in ascending order, and use at least one noise reduction model ranked first as at least one candidate noise reduction model corresponding to the test speech signal sample set; extract the intersection noise reduction model from the candidate noise reduction models corresponding to each test speech signal sample set, and use the intersection noise reduction model as the clustered noise reduction model; wherein the candidate noise reduction model as the intersection noise reduction model corresponds to multiple test speech signal sample sets.

[0143] In some embodiments, the training module 2554 is also used to cluster multiple constraints as original constraints to obtain at least one cluster constraint; obtain a test speech signal sample set corresponding to the multiple original constraints one by one, and a noise reduction model corresponding to the multiple original constraints one by one; fuse the test speech signal sample set corresponding to the multiple original constraints one by one to obtain a test speech signal sample set corresponding to the cluster constraint; perform noise reduction on the test speech signal samples in the test speech signal sample set corresponding to the cluster constraint through multiple noise reduction models to obtain noise reduction results corresponding to the multiple noise reduction models; obtain the minimum mean square error of each noise reduction result, and determine the noise reduction model corresponding to the smallest minimum mean square error as the cluster noise reduction model.

[0144] In some embodiments, the training module 2554 is also used to obtain multiple noises carrying multiple attribute information; wherein the attribute information includes: time information of sending or receiving a voice signal, geographical location information of sending or receiving a voice signal, user information of sending or receiving a voice signal, and environmental information when sending or receiving a voice signal; according to the multiple attribute information, the multiple noises are divided and processed to obtain a noise set corresponding to multiple constraint conditions one by one; wherein each constraint condition has the same multiple attribute information; the noise of each noise set is superimposed with a pure voice signal sample to obtain a training voice signal sample set corresponding to each constraint condition.

[0145] In some embodiments, the training module 2554 is also used to obtain the weight of the clean speech signal sample and the weight of the noise; according to the weight of the clean speech signal sample and the weight of the noise, the clean speech signal sample and the noise are weighted to obtain a training speech signal sample; based on the training speech signal sample, the clean speech signal or noise is added to obtain a new training speech signal sample; and according to the training speech signal sample and the new training speech signal sample, a training speech signal sample set is formed.

[0146] In some embodiments, the training module 2554 is also used to perform denoising on the training speech signal samples included in the training speech signal sample set through a denoising model to obtain a denoised speech signal corresponding to the training speech signal sample; determine the error between the denoised speech signal corresponding to the training speech signal sample and the pure speech signal sample, and substitute the error into the loss function of the denoising model; determine the parameter change value of the denoising model when the loss function obtains the minimum value based on the learning rate of the denoising model, and update the parameters of the denoising model based on the parameter change value.

[0147] In some embodiments, the training module 2554 is also used to extract speech features from training speech signal samples; perform a first fully-connected processing on the speech features through the first fully-connected layer of the speech endpoint network to obtain a first fully-connected processing result; perform speech endpoint detection processing on the first fully-connected result through the first gated loop unit of the speech endpoint network to obtain a speech endpoint detection result; use the speech features, the first fully-connected processing result and the speech endpoint detection result as input, and predict the noise spectrum characteristics of the training speech signal samples through the second gated loop unit of the noise spectrum prediction network; use the noise spectrum characteristics, the speech endpoint detection result and the speech features as input, and predict the gain of the corresponding training speech signal sample through the third gated loop unit of the noise spectrum removal network, and apply the gain to the training speech signal sample.

[0148] The following is a description of an exemplary structure of the training device 255-2 for the denoising model based on artificial intelligence provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules in the training device 255-2 of the artificial intelligence-based noise reduction model stored in the memory 250 may include: a training module 2555, which is used to obtain multiple training speech signal samples carrying noise in multiple constraints to form a training speech signal sample set corresponding to the multiple constraints one by one; the training module 2555 is also used to perform noise reduction processing on the training speech signal samples included in the training speech signal sample set corresponding to the multiple constraints through the noise reduction model corresponding to the multiple constraints one by one, so as to obtain a noise-reduced speech signal; the training module 2555 is also used to determine the error between the noise-reduced speech signal and the pure speech signal sample corresponding to the training speech signal sample, and update the parameters of the noise reduction model corresponding to the constraint according to the error; wherein the noise reduction model is used to perform noise reduction processing on the speech signal in the corresponding constraint.

[0149] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned artificial intelligence-based speech noise reduction method or artificial intelligence-based noise reduction model training method of the embodiment of the present application.

[0150] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the present application embodiment, for example, Figures 3A-3C The artificial intelligence-based speech noise reduction method and Figure 3D The training method of the artificial intelligence-based denoising model is shown.

[0151] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0152] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0153] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0154] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0155] To summarize, through the embodiments of the present application, by obtaining the constraints when the speech signal is applied, the corresponding noise reduction model is called according to different constraints to perform noise reduction processing on the speech signal, that is, different suppression measures are adopted according to different constraints, so as to optimize the noise reduction effect in a targeted manner and improve the noise reduction efficiency.

[0156] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A speech noise reduction method based on artificial intelligence, characterized in that: The method comprises: Acquire a speech signal and acquire a constraint condition corresponding to the speech signal; Acquire a set of training speech signal samples corresponding to the plurality of constraint conditions one by one; Based on a training speech signal sample set corresponding to the plurality of constraint conditions one by one, training and obtaining a plurality of noise reduction models corresponding to the plurality of constraint conditions one by one; Performing clustering processing on the multiple noise reduction models to obtain at least one clustered noise reduction model; Wherein, each of the clustering denoising models corresponds to at least one of the constraint conditions; The clustered noise reduction model adapted to the constraint condition is called to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal.

2. The method according to claim 1, characterized in that The obtaining of the constraint condition corresponding to the speech signal includes: Before acquiring the voice signal, acquiring a constraint condition corresponding to the voice signal, the constraint condition including at least one of a planned time and a planned location for voice communication; The calling of the clustered noise reduction model adapted to the constraint condition to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal includes: Before acquiring the speech signal, pre-acquire the clustered noise reduction model adapted to the constraint condition; When the constraint condition is met, the clustered noise reduction model adapted to the constraint condition is automatically called to perform noise reduction processing on the acquired speech signal to obtain a noise-reduced speech signal.

3. The method according to claim 1, characterized in that Before calling the clustered noise reduction model adapted to the constraint condition to perform noise reduction processing on the speech signal, the method includes: Acquire multiple noise reduction models, where the multiple noise reduction models are a set of all noise reduction models that can be called, or a set of noise reduction models that are adapted to the constraint condition; Presenting options of the multiple noise reduction models in a human-computer interaction interface; In response to a selection operation, the selected denoising model of the cluster is used as the denoising model of the cluster adapted to the constraint condition.

4. The method according to claim 1, characterized in that: The obtaining of the constraint condition corresponding to the speech signal includes: Acquire at least one of the following attribute information of the voice signal: time information of sending or receiving the voice signal, geographical location information of sending or receiving the voice signal, user information of sending or receiving the voice signal, and environment information when sending or receiving the voice signal; The constraint condition corresponding to the attribute information of the speech signal is determined according to a preset correspondence relationship between different attribute information and different constraint conditions.

5. The method according to claim 1, characterized in that The noise reduction model includes a voice endpoint network, a noise spectrum estimation network, and a noise spectrum removal network. The calling of the noise reduction model adapted to the constraint condition to perform noise reduction processing on the voice signal to obtain a noise-reduced voice signal includes: Extracting speech features from the speech signal; Performing a first fully connected processing on the voice feature corresponding to the voice signal through the first fully connected layer of the voice endpoint network to obtain a first fully connected processing result corresponding to the voice signal; Performing voice endpoint detection processing on a first fully connected result corresponding to the voice signal through a first gated recurrent unit of the voice endpoint network to obtain a voice endpoint detection result corresponding to the voice signal; Taking the speech feature corresponding to the speech signal, the first fully connected processing result corresponding to the speech signal, and the speech endpoint detection result corresponding to the speech signal as input, predicting the noise spectrum characteristics of the speech signal through the second gated recurrent unit of the noise spectrum prediction network; Taking the noise spectrum characteristics of the speech signal, the speech endpoint detection result corresponding to the speech signal and the speech features corresponding to the speech signal as input, the gain corresponding to the speech signal is predicted by the third gated recurrent unit of the noise spectrum removal network, and the gain corresponding to the speech signal is applied to the speech signal to obtain a denoised speech signal.

6. The method according to claim 1, characterized in that After calling the clustered noise reduction model adapted to the constraint condition to perform noise reduction processing on the speech signal through the clustered noise reduction model to obtain the noise-reduced speech signal, the method further includes: Determining a noise reduction effect parameter of the noise reduction speech signal; When the noise reduction effect parameter is lower than the noise reduction effect parameter threshold, the following processing is performed: Determine the similarity between a plurality of other constraints and the constraint corresponding to the speech signal, and determine the other constraint with the highest similarity, so as to call the clustered denoising model corresponding to the other constraint to perform denoising on the denoised speech signal, and obtain an updated denoised speech signal; The other constraints are different from the constraints corresponding to the speech signal.

7. The method according to claim 1, characterized in that The clustering process of the plurality of denoising models to obtain at least one clustered denoising model comprises: Acquire a test speech signal sample set corresponding to each of the constraint conditions; The following processing is performed for each of the test speech signal sample sets: performing noise reduction processing on the test speech signal samples in the test speech signal sample set by using the multiple noise reduction models to obtain noise reduction results corresponding to the multiple noise reduction models one by one; According to the noise reduction results obtained for each of the test speech signal sample sets and corresponding one-to-one with the multiple noise reduction models, the multiple noise reduction models are clustered to obtain at least one clustered noise reduction model.

8. The method according to claim 7, characterized in that The method further comprises: performing clustering processing on the plurality of noise reduction models according to the noise reduction results obtained for each of the test speech signal sample sets and corresponding to the plurality of noise reduction models one by one, and obtaining at least one clustered noise reduction model, including: The following processing is performed for each of the test speech signal sample sets: determining a plurality of denoising results obtained after the test speech signal samples in the test speech signal sample set are denoised by a plurality of the denoising models; Determining a minimum mean square error of each of the denoising results, wherein the denoising result includes denoised speech signal samples of a plurality of test speech signal samples in the test speech signal sample set; Based on the minimum mean square error of the denoising result, the plurality of denoising models are sorted in ascending order, and at least one denoising model that is sorted first is used as at least one candidate denoising model corresponding to the test speech signal sample set; Extracting an intersection noise reduction model from the candidate noise reduction models corresponding to each of the test speech signal sample sets, so as to use the intersection noise reduction model as a clustered noise reduction model; There are multiple test speech signal sample sets corresponding to the candidate denoising models serving as the intersection denoising model.

9. The method according to claim 1, characterized in that: The clustering process of the plurality of denoising models to obtain at least one clustered denoising model comprises: Clustering the plurality of constraints as original constraints to obtain at least one cluster constraint; Acquire a test speech signal sample set corresponding to the plurality of original constraint conditions one by one, and a noise reduction model corresponding to the plurality of original constraint conditions one by one; Performing fusion processing on the test speech signal sample sets corresponding to the multiple original constraint conditions one by one to obtain the test speech signal sample set corresponding to the cluster constraint condition; By using the plurality of the noise reduction models, the test speech signal samples in the test speech signal sample set corresponding to the clustering constraint condition are subjected to noise reduction processing to obtain noise reduction results corresponding to the plurality of the noise reduction models one by one; The minimum mean square error of each of the denoising results is obtained, and the denoising model corresponding to the smallest minimum mean square error is determined as the denoising model of the cluster.

10. The method according to claim 1, characterized in that The step of obtaining a set of training speech signal samples corresponding to the plurality of constraint conditions comprises: Obtain multiple noises carrying multiple attribute information; The attribute information includes: time information of sending or receiving the voice signal, geographical location information of sending or receiving the voice signal, user information of sending or receiving the voice signal, and environment information of sending or receiving the voice signal; According to the multiple types of attribute information, the multiple noises are divided and processed to obtain a noise set corresponding to the multiple constraint conditions one by one; Each of the constraint conditions has the same multiple attribute information; The noise of each noise set is superimposed with the clean speech signal sample to obtain a training speech signal sample set corresponding to each constraint condition.

11. The method according to claim 10, characterized in that The step of superimposing the noise of each noise set with the clean speech signal sample to obtain a training speech signal sample set corresponding to each constraint condition comprises: Obtaining the weight of the clean speech signal sample and the weight of the noise; According to the weight of the clean speech signal sample and the weight of the noise, weighted processing is performed on the clean speech signal sample and the noise to obtain a training speech signal sample; Adding the clean speech signal or the noise to the training speech signal sample to obtain a new training speech signal sample; The training speech signal sample set is formed according to the training speech signal samples and the new training speech signal samples.

12. The method according to claim 1, characterized in that The training of a plurality of noise reduction models corresponding to the plurality of constraint conditions in one-to-one correspondence based on a training speech signal sample set corresponding to the plurality of constraint conditions in one-to-one correspondence includes: Performing denoising processing on the training speech signal samples included in the training speech signal sample set by using the denoising model to obtain denoised speech signals corresponding to the training speech signal samples; Determine the error between the denoised speech signal and the clean speech signal sample corresponding to the training speech signal sample, and substitute the error into the loss function of the denoising model; The parameter change value of the denoising model when the loss function obtains a minimum value is determined based on the learning rate of the denoising model, and the parameters of the denoising model are updated based on the parameter change value.

13. The method according to claim 12, characterized in that Said Performing noise reduction processing on the training speech signal samples included in the training speech signal sample set by using the noise reduction model includes: Extracting speech features from training speech signal samples; Performing a first fully connected processing on the speech features corresponding to the training speech signal sample through a first fully connected layer of the speech endpoint network to obtain a first fully connected processing result; Performing voice endpoint detection processing on the first fully connected result corresponding to the training voice signal sample through the first gated recurrent unit of the voice endpoint network to obtain a voice endpoint detection result corresponding to the training voice signal sample; Taking the speech features corresponding to the training speech signal samples, the first fully connected processing results corresponding to the training speech signal samples, and the speech endpoint detection results corresponding to the training speech signal samples as input, the noise spectrum characteristics of the training speech signal samples are predicted through the second gated recurrent unit of the noise spectrum prediction network; Taking the noise spectrum characteristics corresponding to the training speech signal sample, the speech endpoint detection results corresponding to the training speech signal sample and the speech features corresponding to the training speech signal sample as input, the gain corresponding to the training speech signal sample is predicted by the third gated recurrent unit of the noise spectrum removal network, and the gain corresponding to the training speech signal sample is applied to the training speech signal sample.

14. A training method for a denoising model based on artificial intelligence, characterized in that: The method comprises: Acquire a plurality of training speech signal samples carrying noise in a plurality of constraint conditions to form a set of training speech signal samples corresponding one-to-one to the plurality of constraint conditions; Based on a training speech signal sample set corresponding to the plurality of constraint conditions one by one, training a plurality of noise reduction models corresponding to the plurality of constraint conditions one by one; Performing clustering processing on the multiple noise reduction models to obtain at least one clustered noise reduction model; Wherein, each of the clustering denoising models corresponds to at least one of the constraint conditions; Performing denoising processing on the training speech signal samples included in the training speech signal sample set corresponding to the plurality of constraint conditions by using at least one of the clustered denoising models to obtain a denoised speech signal; Determining an error between the denoised speech signal and a clean speech signal sample corresponding to the training speech signal sample, and updating parameters of the denoising model corresponding to the constraint condition according to the error; The noise reduction model is used to perform noise reduction processing on the speech signal in the corresponding constraint conditions.

15. A speech noise reduction device based on artificial intelligence, characterized in that: include: An acquisition module, used to acquire a speech signal and acquire a constraint condition corresponding to the speech signal; A training module, used to obtain a set of training speech signal samples corresponding to the plurality of said constraints one by one; based on the set of training speech signal samples corresponding to the plurality of said constraints one by one, train a plurality of noise reduction models corresponding to the plurality of said constraints one by one; cluster the plurality of noise reduction models to obtain at least one clustered noise reduction model; wherein each of the clustered noise reduction models corresponds to at least one of the said constraints; The noise reduction module is used to call the clustered noise reduction model adapted to the constraint condition to perform noise reduction processing on the speech signal to obtain a noise-reduced speech signal.

16. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the artificial intelligence-based speech noise reduction method described in any one of claims 1 to 13 or the artificial intelligence-based noise reduction model training method described in claim 14 when executing the executable instructions stored in the memory.

17. A computer-readable storage medium, characterized in that: Executable instructions are stored for implementing the artificial intelligence-based speech noise reduction method described in any one of claims 1 to 13 or the artificial intelligence-based noise reduction model training method described in claim 14 when executed by a processor.

18. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by the processor, the speech noise reduction method based on artificial intelligence described in any one of claims 1 to 13 or the training method of the noise reduction model based on artificial intelligence described in claim 14 is implemented.

Images