Audio processing method, device, intelligent device and storage medium

By obtaining the characteristic parameters of the audio signal and using the noise optimization model to process it, the problem of noise interference in voice communication is solved, the quality of voice signals is improved, and the effective reduction or elimination of noise signals is achieved.

CN113516992BActive Publication Date: 2024-12-06TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010851540.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-21
Publication Date
2024-12-06
Estimated Expiration
2040-10-08

AI Technical Summary

Technical Problem

During voice communication, noise signals (such as echo and external noise) interfere with each other, causing the quality of voice signals to decline, affecting the communication effect.

Method used

By obtaining the N-dimensional first characteristic parameters and the M-dimensional second characteristic parameters of the to-process audio signal, the noise optimization model (including the R-layer gated loop unit network and activation layer) is called for processing, the P-dimensional gain parameters are obtained, and the audio is given The signal is subjected to gain calculation to reduce or eliminate the influence of the noise signal.

Benefits of technology

It effectively improves the quality of voice interaction, weakens or eliminates the adverse effects of noise signals on audio signals, and improves the clarity of voice signals during communication.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113516992B_ABST
    Figure CN113516992B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide an audio processing method, apparatus, device and computer-readable storage medium. The method may include: obtaining an audio signal to be processed, and processing the audio signal to be processed to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, calling a noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter, and performing gain calculation processing on the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal. The method can be implemented in a terminal or a cloud server or other device, and by performing gain calculation processing on the audio signal to be processed through the gain parameter, the audio signal to be processed can be more effectively optimized and corrected, and the adverse effects of the characteristics of the noise audio signal in the audio signal to be processed on the audio signal to be processed can be reduced or even eliminated, thereby improving the quality of voice interaction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an audio processing method, apparatus, intelligent device and computer-readable storage medium. Background Art

[0002] With the continuous development of communication technology, users can conduct voice communication anytime and anywhere. Since the communication location is not fixed, there may be noise in the surrounding environment during communication, which makes the transmitted audio signal carry some unnecessary signals.

[0003] In one scenario, the non-essential signal mainly refers to the noise signal, which includes the echo signal and the sound signal emitted by other objects (such as the car horn). These noise signals will have an adverse effect on the voice signal during the communication process. For example, the user at the voice signal receiving end cannot hear the voice content of the user at the voice signal sending end, which reduces the quality of voice interaction. Summary of the invention

[0004] The embodiments of the present application provide an audio processing method, apparatus, device, and computer-readable storage medium, which can improve the quality of audio signals to a certain extent.

[0005] On the one hand, an embodiment of the present application provides an audio processing method, the method comprising:

[0006] Acquire an audio signal to be processed, and process the audio signal to be processed to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, where M and N are positive integers;

[0007] Calling the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter, where P is a positive integer;

[0008] Performing gain calculation processing on the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal;

[0009] The noise optimization model includes an R-layer gated recurrent unit network and an activation layer, wherein the activation layer is used to activate the output result of the R-th layer gated recurrent unit network, and R is a positive integer greater than or equal to 2.

[0010] On the other hand, an embodiment of the present application provides an audio processing device, the processing device comprising:

[0011] an acquisition unit, configured to acquire an audio signal to be processed, and process the audio signal to be processed to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, where M and N are positive integers;

[0012] a processing unit, configured to call a noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter, where P is a positive integer; and perform gain calculation processing on the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal;

[0013] The noise optimization model includes three layers of gated recurrent unit networks and one activation layer, and the activation layer is used to activate the output result of the third layer of gated recurrent unit networks.

[0014] Correspondingly, an embodiment of the present application also provides an audio processing device, including a processor, a memory and a communication interface, wherein the processor, the memory and the communication interface are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the above-mentioned audio processing method.

[0015] Accordingly, the present application provides a computer-readable storage medium, which stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the above-mentioned audio processing method.

[0016] Accordingly, the present application provides a computer program product or a computer program, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned audio processing method.

[0017] In an embodiment of the present application, the acquired audio signal to be processed is characterized by the first characteristic parameter and the second characteristic parameter, and the noise optimization model is called to process the first characteristic parameter and the second characteristic parameter to obtain the gain parameter of the audio signal to be processed. The gain calculation processing is performed on the audio signal to be processed by the gain parameter, and the audio signal to be processed can be optimized and corrected more effectively, reducing or even eliminating the adverse effect of the characteristics of the noise audio signal in the audio signal to be processed on the audio signal to be processed, thereby improving the quality of voice interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1aA scene architecture diagram of audio processing provided in an embodiment of the present application;

[0020] Figure 1b An audio processing flow chart provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of an audio processing method provided in an embodiment of the present application;

[0022] Figure 3a A flowchart of another audio processing method provided in an embodiment of the present application;

[0023] Figure 3b A training flow chart of a noise optimization model provided in an embodiment of the present application;

[0024] Figure 4a A flowchart of calling a noise optimization model to process characteristic parameters of an audio signal to be processed provided in an embodiment of the present application;

[0025] Figure 4b A conference session interface diagram provided in an embodiment of the present application;

[0026] Figure 4c A voice conversation interface diagram provided in an embodiment of the present application;

[0027] Figure 4d A waveform diagram of an audio signal to be processed provided in an embodiment of the present application;

[0028] Figure 4e A waveform diagram of a processed audio signal provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of the structure of a smart device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0032] The embodiments of the present application involve artificial intelligence (AI) and machine learning (ML). By combining AI and ML, it is possible to mine and analyze the features in the audio signal, so that the intelligent device can more accurately identify and process the audio signal, thereby determining the features of the noise signal in the audio signal, so as to reduce or even eliminate the adverse effects of this part of the noise signal on the original audio signal. Among them, AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0033] AI technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic AI technologies generally include technologies such as sensors, dedicated AI chips, cloud computing, distributed storage, processing technologies for large applications, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The embodiments of this application mainly involve language processing technology.

[0034] ML is a multi-disciplinary cross-discipline involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. ML is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. ML and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The embodiments of the present application mainly involve training the initial model with sample data to obtain a trained noise optimization model.

[0035] See also Figure 1a , Figure 1a This is a scene architecture diagram of an audio processing provided by an embodiment of the present application. Figure 1aAs shown, the scenario architecture diagram includes a terminal device 101, a terminal device 102 and a server 103. Among them, the terminal device 101 is a device used by the sender of the audio signal, and the terminal device 102 is a device used by the receiver of the audio signal; in some embodiments, the terminal device 102 can also send an audio signal to the terminal device 101, and accordingly, the terminal device 101 can also receive and play the audio signal sent by the terminal device 102. The number of terminal devices 101 and 102 can be one or more, and the forms of terminal devices 101 and 102 are only used for example. Terminal devices 101 and 102 can include but are not limited to: smart phones (such as Android phones, iOS phones, etc.), tablet computers, portable personal computers, mobile Internet devices (Mobile Internet Devices, referred to as MID), voice collectors (players) and other devices with voice playback and collection functions.

[0036] The server 103 is an audio signal transfer device. The server 103 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device 101, the terminal device 102 and the server 103 can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0037] Figure 1b This is an audio processing flow chart provided in an embodiment of the present application. Figure 1bAs shown, the audio processing flow mainly includes: the terminal device 101 collects the mixed audio signal of the audio signal sender (i.e., obtains the audio signal to be processed), and the mixed audio signal includes a speech audio signal and a noise audio signal; (1) processes the audio signal to be processed to obtain the N-dimensional first feature parameter and the M-dimensional second feature parameter of the audio signal to be processed; (2) calls the noise optimization model to process the N-dimensional first feature parameter and the M-dimensional second feature parameter to obtain the P-dimensional gain parameter; (3) performs gain calculation processing on the audio signal to be processed and the P-dimensional gain parameter (such as multiplying the frequency band after the audio signal to be processed is divided by the gain parameter of the corresponding dimension) to obtain a processed audio signal, which weakens or eliminates the signal corresponding to the noise part in the audio signal to be processed, and can be transmitted to one or more audio signal receivers through the server 103, or can be directly sent to the audio signal receiver by the audio signal sender, that is, the terminal device 101 directly sends the audio signal to the terminal device 102. Among them, the noise optimization model includes an R-layer gated recurrent unit network and an activation layer, and the activation layer is used to activate the output result of the R-th layer gated recurrent unit network. It should be noted that the above steps (1) to (3) can be executed by the terminal device 101 or the terminal device 102 (the terminal device is installed with an application designed based on the noise optimization model), or by the server 103.

[0038] In one embodiment, the N-dimensional first feature parameters include one or more of the following: the i-th dimension first feature parameter in the N-dimensional first feature parameters is used to characterize the characteristics of the speech audio signal (i.e., the frequency band corresponding to the i-th dimension first feature parameter only contains human voice); the j-th dimension first feature parameter in the N-dimensional first feature parameters is used to characterize the characteristics of the noise audio signal (i.e., the frequency band corresponding to the j-th dimension first feature parameter only contains noise); the k-th dimension first feature parameter in the N-dimensional first feature parameters is used to characterize the characteristics of the human voice audio signal and the noise audio signal (i.e., the frequency band corresponding to the k-th dimension first feature parameter contains both human voice and noise). For example, assuming that the i-th dimension gain parameter in the N-dimensional first feature parameters characterizes the correction coefficient corresponding to the frequency band containing only human voice audio signals, and the j-th dimension gain parameter characterizes the correction coefficient corresponding to the frequency band containing only noise audio signals; then the i-th dimension first feature parameter also corresponds to a feature representing the human voice audio signal, and the j-th dimension first feature parameter corresponds to a feature representing the noise signal. Furthermore, during calculation, if the i-th dimension gain parameter in the obtained P-dimensional gain parameter is 1 and the j-th dimension gain parameter is 0.01, then the gain calculation processing of the processed audio signal and the P-dimensional gain parameter means: multiplying the power spectrum of the frequency band corresponding to the j-th dimension first characteristic parameter in the N-dimensional first characteristic parameter by the j-th dimension gain parameter 0.01 in the P-dimensional gain parameter to obtain a new power spectrum. It can be understood that since the power spectrum of the frequency band to which the noise audio belongs is multiplied by the corresponding gain parameter 0.01, the noise power is greatly reduced, while the power spectrum of the frequency band to which the human voice audio belongs is multiplied by the corresponding gain parameter 1, the human voice power remains unchanged, so the influence of the noise audio signal on the human voice audio signal after the operation will be greatly reduced. It should be noted that if the first characteristic parameter of the kth dimension is used to characterize the characteristics of the human voice audio signal and the noise audio signal, the corresponding kth dimension gain parameter can be determined according to the power of the human voice audio signal and the noise audio signal in the frequency band corresponding to the first characteristic parameter of the kth dimension. For example, if the power of the human voice audio signal in the frequency band corresponding to the first characteristic parameter of the kth dimension is 8 times that of the noise audio signal (that is, it means that the noise is small and has little effect on the human voice), the value of the kth dimension gain parameter is determined to be 0.9. Alternatively, the frequency band corresponding to the first characteristic parameter of the kth dimension is further divided to obtain a k1 frequency band containing only human voice audio signals and a k2 frequency band containing only noise audio signals.

[0039] Through practical research, it is found that when the dimension of the characteristic parameters of the input noise optimization model (i.e., N+M) is in the range of [42,70], the noise optimization model can better optimize the audio signal to be processed, enhance the speech signal, and suppress noise; among them, when the dimension of the characteristic parameters of the input noise optimization model is in the range of [45,55], the noise optimization model has a better optimization effect on the audio signal to be processed; further, when the dimension of the characteristic parameters of the input noise optimization model is around 49 dimensions, such as 47 dimensions, 48 ​​dimensions, 49 dimensions, 50 dimensions, and 51 dimensions, the noise optimization model has a more obvious optimization effect on the audio signal to be processed, and the occupied software and hardware resources are more appropriate whether in the model training stage or the stage of optimizing the audio signal to be processed. The solution of this application will be described in detail later, taking N+M=49 as an example.

[0040] In an embodiment of the present application, an audio signal to be processed is obtained, and the audio signal to be processed is processed to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, and a noise optimization model is called to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter, and a gain calculation process is performed on the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal. It can be seen that by performing a gain calculation process on the audio signal to be processed through the gain parameter, the audio signal to be processed can be optimized and corrected more effectively, reducing or even eliminating the adverse effects of the characteristics of the noise audio signal in the audio signal to be processed on the audio signal to be processed, thereby improving the quality of voice interaction.

[0041] See also Figure 2 , Figure 2 This is a flowchart of an audio processing method provided in an embodiment of the present application. The method can be executed by an intelligent device, which can be Figure 1a The terminal device 101, the terminal device 102 or the server 103 shown in the figure, the method of the embodiment of the present application includes the following steps.

[0042] S201: Acquire an audio signal to be processed, and process the audio signal to be processed to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed. The N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter can uniquely characterize the acquired audio signal to be processed, and the duration of the audio signal to be processed can be predetermined. For example, the audio signal to be processed is a signal after frame processing (that is, each frame of audio signal is an audio signal to be processed), then each frame of audio signal will correspond to the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter; for another example, the duration of the audio signal to be processed is an audio signal of 30ms, then every 30ms of audio signal will correspond to the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter; of course, the duration of the audio signal to be processed can also be other values.

[0043] In one embodiment, the smart device can collect the audio signal to be processed through the voice collection device carried by the smart device, for example, the terminal device collects the user's voice through the microphone carried by the terminal device. In another embodiment, the smart device can receive the audio signal to be processed sent by other devices through the communication interface, for example, the server receives the audio signal to be processed sent by the terminal device.

[0044] S202: Call the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter. The noise optimization model includes an R-layer gated recurrent unit network and an activation layer, and the activation layer is used to activate the output result of the R-th layer gated recurrent unit network, where R is a positive integer greater than or equal to 2, for example, R=3. The gain parameter is used to reduce or eliminate the noise audio signal in the audio signal to be processed. In one embodiment, P=N, that is, after the noise optimization model processes the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter, the gain parameter of the frequency band corresponding to each dimensional first characteristic parameter is obtained. Based on the noise optimization model, the greater the power of the noise contained in each frequency band, the smaller the value of the desired gain parameter, that is, the value of the desired gain parameter is inversely proportional to the power of the noise, so as to minimize or even eliminate the noise in the audio signal to be processed. The value range of each gain parameter is [0,1], and the parameter value corresponding to the frequency band to which speech (usually referring to human voice) belongs in the gain parameter is 1 or close to 1, and the parameter value corresponding to the frequency band to which noise belongs is 0 or close to 0. For example, assuming that frequency band 1 is a frequency band to which speech belongs, and frequency band 2 is a frequency band to which noise belongs, the value of the gain parameter of frequency band 1 is 1 or close to 1, and the value of the gain parameter of frequency band 2 is 0 or close to 0. It can be understood that if the values ​​of the P-dimensional gain parameters are all greater than the gain parameter threshold, it means that the noise in the audio signal to be processed is small or there is no noise; if the values ​​of the P-dimensional gain parameters are all less than the gain parameter threshold, it means that the noise in the audio signal to be processed is large or the audio signal to be processed does not contain speech.

[0045] In another embodiment, P is not equal to N. For example, assuming that the frequency bands corresponding to the N-dimensional first characteristic parameters include frequency band 1, which contains neither speech nor noise, then P=N-1, that is, the P-dimensional gain parameters do not include the gain parameters corresponding to frequency band 1.

[0046] S203: Perform gain calculation processing on the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal. The smart device modifies the power spectrum of the frequency band to which the noise in the audio signal to be processed belongs through the gain parameter to minimize or even eliminate the noise in the audio signal to be processed, thereby obtaining a processed audio signal.

[0047] In one embodiment, after the smart device collects the audio signal to be processed through the audio collection device (such as a microphone) carried by the smart device, the audio signal to be processed is processed by executing the above steps S201-S203, and the processed audio signal is sent to one or more terminal devices participating in the communication. In another embodiment, the smart device obtains the audio signal to be processed sent by the first terminal device to the second terminal device, processes the audio signal to be processed by executing the above steps S201-S203, and sends the processed audio signal to the second terminal device. In yet another embodiment, the smart device receives the audio signal to be processed sent by the terminal device, processes the audio signal to be processed by executing the above steps S201-S203, and plays the processed audio signal.

[0048] In an embodiment of the present application, the acquired audio signal to be processed is characterized by the first characteristic parameter and the second characteristic parameter, and the noise optimization model is called to process the first characteristic parameter and the second characteristic parameter to obtain the gain parameter of the audio signal to be processed. The gain calculation processing is performed on the audio signal to be processed by the gain parameter, and the audio signal to be processed can be optimized and corrected more effectively, reducing or even eliminating the adverse effect of the characteristics of the noise audio signal in the audio signal to be processed on the audio signal to be processed, thereby improving the quality of voice interaction.

[0049] See also Figure 3a , Figure 3a A flowchart of another audio processing method provided in an embodiment of the present application. The method can be executed by an intelligent device, which can be Figure 1a The terminal device 101, the terminal device 102 or the server 103 shown in the figure, the method of the embodiment of the present application includes the following steps.

[0050] S301: Acquire the audio signal to be processed. The specific implementation of step S301 can be found in Figure 2In one embodiment, the smart device detects the sampling rate of the audio signal to be processed, and if the sampling rate of the audio signal to be processed is detected to be the target sampling rate, the following step S302 is triggered to be executed, and the target sampling rate can be a higher sampling rate, such as a 32K sampling rate. Specifically, in S301, for the 16K bandwidth signal corresponding to the speech with a sampling rate of 32K, the audio processing method of the present application can be adopted, and the noise optimization model in the embodiment of the present application can be used for analysis, so as to achieve language signal enhancement and noise suppression. For the 8K bandwidth signal corresponding to the speech with a sampling rate of 16K, other existing methods can be used for speech enhancement processing, wherein the 16K bandwidth signal mentioned above refers to a speech signal with a cutoff frequency of 16K, and the sampling rate of the speech signal is generally required to be 2 times or more of the cutoff frequency. Therefore, for the detected speech signal with a cutoff frequency of 8K, the sampling rate is 16K, and for the speech signal with a cutoff frequency of 16K, the sampling rate is 32K. The higher the sampling rate, the richer the details of the speech signal, and the improved scheme of the present application is required to perform speech signal enhancement processing and better noise suppression.

[0051] Optionally, if it is detected that the sampling rate of the audio signal to be processed is different from the target sampling rate, other preset speech enhancement models may be used to process the audio signal to be processed.

[0052] S302: Divide the audio signal to be processed into N frequency bands, and perform operations on each frequency band to obtain the N-dimensional first characteristic parameter of the audio signal to be processed. The frequency band refers to the bandwidth occupied by the signal, that is, the maximum bandwidth of the signal that can effectively pass through the channel. The intelligent device divides the audio signal to be processed in the order of low to high frequency to obtain N frequency bands corresponding to the audio signal to be processed, each frequency band includes at least one frequency point, and the maximum frequency of the i-th frequency band in the N frequency bands corresponding to the audio signal to be processed is lower than the lowest frequency of the i+1-th frequency band, and the number of frequency points in the i-th frequency band is less than the number of frequency points in the i+1-th frequency band. In one embodiment, the intelligent device divides the audio signal to be processed into 29 frequency bands, that is, N=29; and calculates the frequency cepstrum coefficients corresponding to each frequency band to obtain the N-dimensional first characteristic parameter; wherein the frequency cepstrum coefficients can be Bark frequency cepstrum coefficients (BFCC), Mel frequency cepstrum coefficients (MFCC), etc.

[0053] S303: Process the audio signal to be processed to obtain the M-dimensional second characteristic parameters of the audio signal to be processed. The M-dimensional second characteristic parameters include M-2-dimensional low-frequency characteristic parameters, filter characteristic parameters and energy parameters; wherein the M-2-dimensional low-frequency characteristic parameters are obtained by performing a differential operation on the first j-dimensional first characteristic parameters of the N-dimensional first characteristic parameters, and the first j-dimensional first characteristic parameters are characteristic parameters used to represent the low-frequency signal of the audio signal to be processed, j is a positive integer, and j is less than N. Specifically, the intelligent device calculates the discrete cosine transform (DCT) values ​​of the first-order difference, second-order difference and correlation coefficient of the first j-dimensional first characteristic parameters to obtain the M-2-dimensional low-frequency characteristic parameters.

[0054] The filtering characteristic parameters are obtained by processing the audio signal to be processed using a filter. In one embodiment, a pitch filter is used to filter the audio signal to be processed to obtain a pitch characteristic value corresponding to the audio signal to be processed. The energy characteristic parameters are obtained by calculating the audio signal to be processed. In one embodiment, the audio signal to be processed is subjected to a fast Fourier transform (FFT) to obtain the energy parameters corresponding to the audio signal to be processed. In another embodiment, the audio signal to be processed is subjected to voice activity detection (VAD) to obtain the energy parameters corresponding to the audio signal to be processed.

[0055] In one embodiment, M=20, that is, the dimension of the second characteristic parameter is 20; wherein, the first 6 dimensions of BFCC in the N-dimensional first characteristic parameter are characteristic parameters for representing the low-frequency signal of the audio signal to be processed, and the DCT values ​​of the 6-dimensional first-order difference, the 6-dimensional second-order difference and the 6-dimensional correlation coefficient can be calculated through the first 6-dimensional BFCC to obtain 18 (M-2)-dimensional low-frequency characteristic parameters. In addition, it also includes a 1-dimensional filtering characteristic parameter and a 1-dimensional energy characteristic parameter.

[0056] S304: Call the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter. In one embodiment, based on the above-mentioned N=29-dimensional first characteristic parameter and M=20-dimensional second characteristic parameter, a 49-dimensional characteristic parameter input to the noise optimization model can be obtained, and the corresponding P-dimensional gain parameter is obtained through the noise optimization model. P is the gain parameter corresponding to N, and in one embodiment, P=N.

[0057] Figure 3b A training flow chart of a noise optimization model provided in an embodiment of the present application, such as Figure 3bAs shown, the noise optimization model includes three layers of gated recurrent units, that is, R=3. The noise optimization model is executed before calling the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain the P-dimensional gain parameter. Figure 3b The results are obtained from steps S3041 to S3043 as shown.

[0058] In one implementation, S3041: use the initial model to process the N-dimensional first sample feature parameters and the M-dimensional second sample feature parameters of the speech sample data to obtain a P-dimensional initial gain parameter. The audio signal included in the speech sample data is a mixed signal, that is, a portion of the audio signal included in the speech sample data is a pure speech audio signal, a portion of the audio signal is a noise audio signal, and another portion of the audio signal is an audio signal that contains both a speech audio signal and a noise audio signal. The specific implementation of step S3041 can be referred to Figure 2 The implementation of step S202 will not be repeated here.

[0059] S3042: Compare the P-dimensional initial gain parameter and the actual gain of the speech sample data through the cost function. The actual gain is calculated by the speech sample data and the clean speech data corresponding to the speech sample data, that is, the audio signal obtained by performing gain calculation processing on the speech sample data and the actual gain is the same as the audio signal of the clean speech data, or is close to the audio signal of the clean speech data. The cost function is used to adjust the parameters in the initial model to appropriate values ​​so that the P-dimensional initial gain parameter is the same as the actual gain, or is close to the actual gain. The cost function includes: Mean Squared Error (MSE), Mean Absolute Error (MAE), Cross Entry cost function (CE), etc.; for example, the smart device compares the P-dimensional initial gain parameter with the actual gain by MSE.

[0060] S3043: Adjust the parameters in the initial model according to the comparison result to obtain the noise optimization model. The smart device adjusts the parameters in the initial model (such as activation function, number of nodes, etc.) according to the P-dimensional initial gain parameter and the actual gain to obtain the noise optimization model.

[0061] Figure 4a A flowchart of calling a noise optimization model to process characteristic parameters of an audio signal to be processed is provided in an embodiment of the present application. Figure 4aAs shown, the noise optimization model includes three layers of gated recurrent unit networks and one activation layer. First, the N-dimensional first feature parameter and the M-dimensional second feature parameter are input into the first layer of the gated recurrent unit network of the noise optimization model, and then the N-dimensional first feature parameter, the M-dimensional second feature parameter and the output of the first layer of the gated recurrent unit network are input into the second layer of the gated recurrent unit network, then the N-dimensional first feature parameter, the M-dimensional second feature parameter, the output of the first layer of the gated recurrent unit network and the output of the second layer of the gated recurrent unit network are input into the third layer of the gated recurrent unit network, and finally the output of the third layer of the gated recurrent unit network is input into the activation layer for activation, and P is the gain parameter.

[0062] In one embodiment, the three-layer gated recurrent unit network includes an activation function, which is used to add nonlinear factors to the model and mine the hidden logical relationships in the data. The activation functions in each layer of the gated recurrent unit network can be the same or different, and this application is not limited to this. Activation functions include: sigmoid function, hyperbolic tangent (tanh) function, linear rectifier (Rectified Linear Unit, ReLU) function, Gaussian error (Gaussian Error Linear Units, GELU) function, etc. For example, the first layer of the gated recurrent unit network is composed of 40 computing nodes, and the activation function included in the first layer of the gated recurrent unit network is the tanh function; the second layer of the gated recurrent unit network is composed of 50 computing nodes, and the activation function included in the second layer of the gated recurrent unit network is the ReLU function; the third layer of the gated recurrent unit network is composed of 100 computing nodes, and the activation function included in the third layer of the gated recurrent unit network is the tanh function; the activation layer is composed of 29 computing nodes, and the activation layer is constructed based on the sigmoid function.

[0063] S305: Calculate the power spectrum of each frequency band with the gain parameter of the corresponding dimension to obtain the optimized power spectrum corresponding to each frequency band. The power spectrum is used to indicate the signal power within a unit frequency band. In one embodiment, the intelligent device calculates the power spectrum of the i-th frequency band among the N frequency bands corresponding to the audio signal to be processed with the i-th dimension gain parameter among the P-dimensional gain parameters to obtain the optimized power spectrum corresponding to the i-th frequency band. The greater the power of the noise contained in each frequency band, the smaller the value of the desired gain parameter, that is, the value of the desired gain parameter is inversely proportional to the power of the noise, so as to minimize or even eliminate the noise in the audio signal to be processed. The value range of each gain parameter is [0,1], and the parameter value of the gain parameter corresponding to the frequency band to which the speech belongs is 1 or close to 1, and the parameter value corresponding to the frequency band to which the noise belongs is 0 or close to 0. It can be seen that by calculating the power spectrum of each frequency band with the gain parameter of the corresponding dimension, the noise in the audio signal to be processed can be reduced or eliminated, and the quality of the audio signal can be improved.

[0064] S306: Perform signal conversion processing on the optimized power spectrum corresponding to each frequency band to obtain a processed audio signal. In one implementation, the smart device performs an inverse FFT transform on the optimized power spectrum corresponding to each frequency band to obtain a processed audio signal.

[0065] Figure 4b A conference session interface diagram provided in an embodiment of the present application. Figure 4b As shown, the audio signal to be processed is obtained by intercepting the real-time communication audio signal in the time domain. Figure 2 Steps S201-S203 in, or Figure 3a After processing the audio signal to be processed in steps S301-S306, the smart device splices the processed audio signals according to the time domain information of the processed audio signals to obtain an optimized audio signal corresponding to the real-time audio signal. For example, the smart device collects a target speech of conference user 1 during the meeting to obtain a corresponding audio signal, divides the audio signal into audio signal 1 to be processed to be processed 10 in time domain order, processes the audio signals 1 to be processed 10 respectively to obtain N-dimensional first feature parameters and M-dimensional second feature parameters corresponding to each audio signal to be processed, uses a noise optimization model to process the N-dimensional first feature parameters and M-dimensional second feature parameters corresponding to each audio signal to be processed to obtain a P-dimensional gain parameter corresponding to each audio signal to be processed, uses the P-dimensional gain parameters corresponding to each audio signal to be processed to perform gain operation processing on the corresponding audio signals to be processed to obtain processed audio signals 1 to processed audio signals 10, splices the processed audio signals 1 to processed audio signals 10 according to the time domain information of the processed audio signals to obtain an optimized audio signal corresponding to the target speech, and sends the optimized audio signal to other users participating in the meeting.

[0066] Figure 4c A voice conversation interface diagram provided in an embodiment of the present application. Figure 4c As shown, the audio signal to be processed is recorded by the user. If the duration of the recorded voice is less than the division threshold, the smart device uses the audio signal corresponding to the entire voice as the audio signal to be processed. Figure 2 Steps S201-S203 in, or Figure 3a After the audio signal to be processed is processed in steps S301-S306, the smart device saves the processed audio signal or sends the processed audio signal to the corresponding terminal device according to the identifier of the session user.

[0067] Figure 4dThe waveform diagram of an audio signal to be processed provided in an embodiment of the present application is shown in FIG. Figure 4d As shown, the audio signal to be processed includes a speech audio signal and an excess noise audio signal, and the waveform is proportional to the energy of the signal, that is, the larger the waveform amplitude, the larger the energy of the corresponding audio signal. Figure 2 or Figure 3a After the audio signal to be processed is processed by the audio processing method, the processed audio signal can be obtained. The waveform diagram of the processed audio signal is as follows Figure 4e As shown by Figure 4e It can be seen that the energy of the redundant noise audio signal is greatly suppressed or even completely eliminated (i.e. the waveform amplitude is significantly reduced or disappears), while the voice audio signal is almost unaffected (i.e. the waveform changes slightly or remains unchanged). It can be seen that the use of processed audio signals can suppress noise to a certain extent, thereby improving communication quality.

[0068] In the embodiment of the present application, the audio signal to be processed is divided into multiple frequency bands, and the characteristic parameters corresponding to each frequency band are calculated, and the noise optimization model is called to process the characteristic parameters corresponding to each frequency band to obtain the gain parameters corresponding to each frequency band. By calculating the power spectrum of each frequency band and the gain parameters of the corresponding dimension, the noise in the audio signal to be processed can be reduced or eliminated, and the quality of the audio signal can be improved. In addition, the noise optimization model is composed of an R-layer (such as R=3) gated recurrent unit network and an activation layer, which has low complexity, thereby ensuring the real-time performance of the speech optimization process.

[0069] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0070] See also Figure 5 , Figure 5 This is a structural diagram of an audio processing device provided in an embodiment of the present application. The device can be mounted on the smart device in the above method embodiment. The smart device can specifically be Figure 1a The terminal device 101, the terminal device 102 or the server 103 shown in FIG. Figure 5 The audio processing device shown can be used to perform the above Figure 2 , Figure 3a and Figure 3b Part or all of the functions of the method embodiment described. The detailed description of each unit is as follows:

[0071] The processing unit 502 is used to obtain an audio signal to be processed;

[0072] An acquiring unit 501 is configured to process the audio signal to be processed to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, where M and N are positive integers;

[0073] Calling the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter, where P is a positive integer;

[0074] Performing gain calculation processing on the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal;

[0075] The noise optimization model includes an R-layer gated recurrent unit network and an activation layer, wherein the activation layer is used to activate the output result of the R-th layer gated recurrent unit network, and R is a positive integer greater than or equal to 2.

[0076] In one embodiment, the processing unit 502 is specifically configured to: process the audio signal to be processed to obtain an N-dimensional first characteristic parameter of the audio signal to be processed;

[0077] Dividing the audio signal to be processed to obtain N frequency bands, the number of frequency points in the i-th frequency band is less than the number of frequency points in the i+1-th frequency band, i is a positive integer, and i is less than or equal to N;

[0078] An operation is performed on each frequency band respectively to obtain the N-dimensional first characteristic parameter of the audio signal to be processed.

[0079] In one embodiment, the processing unit 502 is specifically configured to: process the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal;

[0080] Calculating the power spectrum of the i-th frequency band in the N frequency bands and the i-th dimension gain parameter in the P-dimensional gain parameters to obtain an optimized power spectrum corresponding to the i-th frequency band;

[0081] The optimized power spectrum corresponding to each frequency band is subjected to signal conversion processing to obtain a processed audio signal.

[0082] In one embodiment, the M-dimensional second characteristic parameters include M-2-dimensional low-frequency characteristic parameters, filtering characteristic parameters and energy parameters;

[0083] The M-2-dimensional low-frequency feature parameters are obtained by calculating the first j-dimensional first feature parameters of the N-dimensional first feature parameters, and the first j-dimensional first feature parameters are used to represent the characteristics of the low-frequency signal of the audio signal to be processed, j is a positive integer, and j is less than N;

[0084] The filtering characteristic parameter is obtained by processing the audio signal to be processed by using a filter, and the energy parameter is obtained by calculating the audio signal to be processed.

[0085] In one embodiment, the noise optimization model includes three layers of gated recurrent units;

[0086] The input of the first layer of the gated recurrent unit network of the noise optimization model is the N-dimensional first feature parameter and the M-dimensional second feature parameter;

[0087] The input of the second layer gated recurrent unit network of the noise optimization model is the output of the first layer gated recurrent unit network, the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter;

[0088] The input of the third-layer gated recurrent unit network of the noise optimization model is the output of the first-layer gated recurrent unit network, the output of the second-layer gated recurrent unit network, the N-dimensional first feature parameter and the M-dimensional second feature parameter.

[0089] In one embodiment, the noise optimization model is trained based on speech sample data, the audio signal included in the sample data is a mixed signal, and the noise optimization model is called to process the N-dimensional first feature parameter and the M-dimensional second feature parameter to obtain the P-dimensional gain parameter. The processing unit 502 is also used to:

[0090] Using the initial model to process the N-dimensional first sample feature parameters and the M-dimensional second sample feature parameters of the speech sample data to obtain a P-dimensional initial gain parameter;

[0091] Comparing the P-dimensional initial gain parameter with the actual gain of the speech sample data by using a cost function, wherein the actual gain is calculated by using the speech sample data and the clean speech data corresponding to the speech sample data;

[0092] The parameters in the initial model are adjusted according to the comparison result to obtain the noise optimization model.

[0093] In one embodiment, the processing unit 502 is specifically configured to:

[0094] The real-time communication audio signal is acquired by the acquisition unit 501, and the real-time audio signal is intercepted in the time domain according to the segment interception threshold to obtain an audio segment signal set, and each audio segment signal in the audio segment signal set is used as an audio signal to be processed;

[0095] The processing unit 502 is further configured to:

[0096] The processed audio signals are spliced ​​according to the time domain information of the processed audio signals to obtain an optimized audio signal corresponding to the real-time audio signal.

[0097] In one embodiment, after acquiring the audio signal to be processed and before processing the audio signal to be processed, the processing unit 502 is further configured to:

[0098] Detecting the sampling rate of the audio signal to be processed;

[0099] If the detected sampling rate is the target sampling rate, the processing of the audio signal to be processed is triggered to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed.

[0100] According to one embodiment of the present application, Figure 2 , Figure 3a and Figure 3b Some of the steps involved in the audio processing method shown can be represented by Figure 5 The audio processing device shown in the figure is executed by each unit. For example, Figure 2 The step S201 shown in FIG. 1 can be performed by Figure 5 The acquisition unit 501 and the processing unit 502 shown in the figure are executed together, and step S202 and step S203 can be performed by Figure 5 Processing unit 502 is shown executing. Figure 3a The step S301 shown in FIG. Figure 5 The acquisition unit 501 shown in FIG. 1 is executed, and steps S302 to S306 can be performed by Figure 5 Processing unit 502 is shown executing. Figure 3b Steps S3041 to S3043 shown in FIG. 3 can be performed by Figure 5 Processing unit 502 is shown executing. Figure 5 The various units in the audio processing device shown can be separately or completely combined into one or several other units to form, or one (some) of the units can also be split into multiple smaller units in function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the audio processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0101] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM) and other processing elements and storage elements. Figure 2 , Figure 3a and Figure 3b A computer program (including program code) for each step involved in the corresponding method shown in Figure 5 The audio processing device shown in and the audio processing method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer readable recording medium, and loaded into the above-mentioned computing device through the computer readable recording medium, and run therein.

[0102] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the audio processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the audio processing device in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0103] See also Figure 6 , Figure 6 A schematic diagram of the structure of an intelligent device provided in an embodiment of the present application, the intelligent device at least includes a processor 601, a communication interface 602 and a memory 603. Among them, the processor 601, the communication interface 602 and the memory 603 can be connected via a bus or other means. Among them, the processor 601 (or central processing unit (CPU)) is the computing core and control core of the terminal, which can parse various instructions in the terminal and process various data of the terminal. For example, the CPU can be used to parse the power on and off instructions sent by the user to the terminal, and control the terminal to perform power on and off operations; for example, the CPU can transmit various types of interactive data between the internal structures of the terminal, and so on. The communication interface 602 can optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 601; the communication interface 602 can also be used for the transmission and interaction of internal data of the terminal. The memory 603 (Memory) is a memory device in the terminal for storing programs and data. It can be understood that the memory 603 here can include both the built-in memory of the terminal and the extended memory supported by the terminal. The memory 603 provides a storage space, which stores the operating system of the terminal, including but not limited to: Android system, iOS system, Windows Phone system, etc., which is not limited in this application.

[0104] In the embodiment of the present application, the processor 601 is used to perform the following operations by running the executable program code in the memory 603:

[0105] Acquire an audio signal to be processed through the communication interface 602, and process the audio signal to be processed to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, where M and N are positive integers;

[0106] Calling the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain a P-dimensional gain parameter, where P is a positive integer;

[0107] Performing gain calculation processing on the audio signal to be processed and the P-dimensional gain parameter to obtain a processed audio signal;

[0108] The noise optimization model includes an R-layer gated recurrent unit network and an activation layer, wherein the activation layer is used to activate the output result of the R-th layer gated recurrent unit network, and R is a positive integer greater than or equal to 2.

[0109] As an optional embodiment, the processor 601 processes the audio signal to be processed, and a specific embodiment of obtaining the N-dimensional first characteristic parameter of the audio signal to be processed is:

[0110] Dividing the audio signal to be processed to obtain N frequency bands, the number of frequency points in the i-th frequency band is less than the number of frequency points in the i+1-th frequency band, i is a positive integer, and i is less than or equal to N;

[0111] An operation is performed on each frequency band respectively to obtain the N-dimensional first characteristic parameter of the audio signal to be processed.

[0112] As an optional embodiment, the processor 601 processes the audio signal to be processed and the P-dimensional gain parameter to obtain a specific embodiment of the processed audio signal:

[0113] Calculating the power spectrum of the i-th frequency band in the N frequency bands and the i-th dimension gain parameter in the P-dimensional gain parameters to obtain an optimized power spectrum corresponding to the i-th frequency band;

[0114] The optimized power spectrum corresponding to each frequency band is subjected to signal conversion processing to obtain a processed audio signal.

[0115] As an optional embodiment, the M-dimensional second characteristic parameter includes an M-2-dimensional low-frequency characteristic parameter, a filtering characteristic parameter and an energy parameter;

[0116] The M-2-dimensional low-frequency feature parameters are obtained by calculating the first j-dimensional first feature parameters of the N-dimensional first feature parameters, and the first j-dimensional first feature parameters are used to represent the characteristics of the low-frequency signal of the audio signal to be processed, j is a positive integer, and j is less than N;

[0117] The filtering characteristic parameter is obtained by processing the audio signal to be processed by using a filter, and the energy parameter is obtained by calculating the audio signal to be processed.

[0118] As an optional embodiment, the noise optimization model includes three layers of gated recurrent units;

[0119] The input of the first layer of the gated recurrent unit network of the noise optimization model is the N-dimensional first feature parameter and the M-dimensional second feature parameter;

[0120] The input of the second layer gated recurrent unit network of the noise optimization model is the output of the first layer gated recurrent unit network, the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter;

[0121] The input of the third-layer gated recurrent unit network of the noise optimization model is the output of the first-layer gated recurrent unit network, the output of the second-layer gated recurrent unit network, the N-dimensional first feature parameter and the M-dimensional second feature parameter.

[0122] As an optional embodiment, the noise optimization model is obtained by training according to speech sample data, the audio signal included in the sample data is a mixed signal, and the noise optimization model is called to process the N-dimensional first feature parameter and the M-dimensional second feature parameter to obtain the P-dimensional gain parameter. The processor 601 is also used to:

[0123] Using the initial model to process the N-dimensional first sample feature parameters and the M-dimensional second sample feature parameters of the speech sample data to obtain a P-dimensional initial gain parameter;

[0124] Comparing the P-dimensional initial gain parameter with the actual gain of the speech sample data by using a cost function, wherein the actual gain is calculated by using the speech sample data and the clean speech data corresponding to the speech sample data;

[0125] The parameters in the initial model are adjusted according to the comparison result to obtain the noise optimization model.

[0126] As an optional embodiment, a specific embodiment in which the processor 601 obtains the audio signal to be processed through the communication interface 602 is:

[0127] Acquire a real-time communication audio signal, and intercept the real-time audio signal in the time domain according to a segment interception threshold to obtain an audio segment signal set, and use each audio segment signal in the audio segment signal set as an audio signal to be processed;

[0128] The processor 601 is further configured to:

[0129] The processed audio signals are spliced ​​according to the time domain information of the processed audio signals to obtain an optimized audio signal corresponding to the real-time audio signal.

[0130] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the smart device provided in the embodiments of the present application are similar to the principles and beneficial effects of solving the problems by the audio processing method in the method embodiments of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0131] An embodiment of the present application also provides a computer-readable storage medium, in which one or more instructions are stored. The one or more instructions are suitable for being loaded by a processor and executing the audio processing method described in the above method embodiment.

[0132] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the audio processing method described in the above method embodiment.

[0133] The embodiment of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned audio processing method.

[0134] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0135] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0136] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the readable storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0137] What is disclosed above is only a preferred embodiment of the present application, and it certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present application are still within the scope of the invention.

Claims

1. An audio processing method, It is characterized in that The method comprises: Acquire a real-time communication audio signal, and intercept the real-time communication audio signal in the time domain according to a segment interception threshold to obtain an audio segment signal set, and use each audio segment signal in the audio segment signal set as an audio signal to be processed; the real-time communication audio signal includes an audio signal collected in a conference scene; Detecting the sampling rate of each section of the audio signal to be processed; If the detected sampling rate is 32K sampling rate, the audio signal to be processed is divided to obtain N frequency bands; the number of frequency points in the i-th frequency band is less than the number of frequency points in the i+1-th frequency band, i is a positive integer, and i is less than or equal to N; Performing operations on each frequency band respectively to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, where M and N are positive integers; Calling the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain an N-dimensional gain parameter, where N is a positive integer; wherein a frequency band corresponding to each dimensional first characteristic parameter corresponds to a one-dimensional gain parameter; and, if the power of the noise contained in the i-th frequency band among the N frequency bands is greater, the value of the gain parameter corresponding to the i-th frequency band is smaller; Calculating the power spectrum of the i-th frequency band in the N frequency bands and the i-th dimension gain parameter in the N-dimensional gain parameters to obtain an optimized power spectrum corresponding to the i-th frequency band; Performing signal conversion processing on the optimized power spectrum corresponding to each frequency band to obtain a processed audio signal; splicing the processed audio signals according to the time domain information of the processed audio signals to obtain an optimized audio signal corresponding to the real-time communication audio signal; Wherein, the noise optimization model includes three layers of gated recurrent units and one activation layer; The input of the first layer of the gated recurrent unit network of the noise optimization model is the N-dimensional first feature parameter and the M-dimensional second feature parameter; The input of the second layer gated recurrent unit network of the noise optimization model is the output of the first layer gated recurrent unit network, the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter; The input of the third layer of the gated recurrent unit network of the noise optimization model is the output of the first layer of the gated recurrent unit network, the output of the second layer of the gated recurrent unit network, the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter; The activation layer is used to activate the output result of the third layer gated recurrent unit network to obtain an N-dimensional gain parameter.

2. The method according to claim 1, It is characterized in that The M-dimensional second characteristic parameters include M-2-dimensional low-frequency characteristic parameters, filtering characteristic parameters and energy parameters; The M-2-dimensional low-frequency feature parameters are obtained by calculating the first j-dimensional first feature parameters of the N-dimensional first feature parameters, and the first j-dimensional first feature parameters are used to represent the characteristics of the low-frequency signal of the audio signal to be processed, j is a positive integer, and j is less than N; The filtering characteristic parameter is obtained by processing the audio signal to be processed by using a filter, and the energy parameter is obtained by calculating the audio signal to be processed.

3. The method according to claim 1, It is characterized in that The noise optimization model is obtained by training based on speech sample data, the audio signal included in the speech sample data is a mixed signal, and the noise optimization model is called to process the N-dimensional first feature parameter and the M-dimensional second feature parameter to obtain the N-dimensional gain parameter. The method also includes: Using the initial model to process the N-dimensional first sample feature parameters and the M-dimensional second sample feature parameters of the speech sample data to obtain an N-dimensional initial gain parameter; Comparing the N-dimensional initial gain parameter and the actual gain of the speech sample data by using a cost function to obtain a comparison result; the actual gain is calculated by using the speech sample data and the clean speech data corresponding to the speech sample data; The parameters in the initial model are adjusted according to the comparison result to obtain the noise optimization model.

4. An audio processing device, It is characterized in that include: an acquisition unit, configured to acquire a real-time communication audio signal, and to intercept the real-time communication audio signal in a time domain according to a segment interception threshold value to obtain an audio segment signal set, and to use each audio segment signal in the audio segment signal set as an audio signal to be processed; the real-time communication audio signal includes an audio signal collected in a conference scene; A processing unit, used for detecting the sampling rate of each section of the audio signal to be processed; The processing unit is further configured to divide the audio signal to be processed into N frequency bands if the detected sampling rate is 32K sampling rate; The number of frequency points in the i-th frequency band is less than the number of frequency points in the i+1-th frequency band, where i is a positive integer and i is less than or equal to N; The processing unit is further used to perform operations on each frequency band respectively to obtain an N-dimensional first characteristic parameter and an M-dimensional second characteristic parameter of the audio signal to be processed, where M and N are positive integers; The processing unit is further used to call the noise optimization model to process the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter to obtain an N-dimensional gain parameter; wherein the frequency band corresponding to each dimensional first characteristic parameter corresponds to the one-dimensional gain parameter; and if the power of the noise contained in the i-th frequency band among the N frequency bands is greater, the value of the gain parameter corresponding to the i-th frequency band is smaller; The processing unit is further used to calculate the power spectrum of the i-th frequency band in the N frequency bands and the i-th dimension gain parameter in the N-dimensional gain parameters to obtain an optimized power spectrum corresponding to the i-th frequency band; The processing unit is further used to perform signal conversion processing on the optimized power spectrum corresponding to each frequency band to obtain a processed audio signal; The processing unit is further used to splice the processed audio signals according to the time domain information of the processed audio signals to obtain an optimized audio signal corresponding to the real-time communication audio signal; Wherein, the noise optimization model includes three layers of gated recurrent units and one activation layer; The input of the first layer of the gated recurrent unit network of the noise optimization model is the N-dimensional first feature parameter and the M-dimensional second feature parameter; The input of the second layer gated recurrent unit network of the noise optimization model is the output of the first layer gated recurrent unit network, the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter; The input of the third layer of the gated recurrent unit network of the noise optimization model is the output of the first layer of the gated recurrent unit network, the output of the second layer of the gated recurrent unit network, the N-dimensional first characteristic parameter and the M-dimensional second characteristic parameter; The activation layer is used to activate the output result of the third layer gated recurrent unit network to obtain an N-dimensional gain parameter.

5. A smart device, It is characterized in that include: storage devices and processors; The storage device stores a computer program; The processor executes a computer program to implement the audio processing method according to any one of claims 1 to 3.

6. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the audio processing method according to any one of claims 1 to 3 is implemented.

7. A computer program product, It is characterized in that The invention comprises a computer program, which implements the audio processing method according to any one of claims 1 to 3 when executed by a processor.