Audio noise reduction method and related apparatus

Abstract

The application discloses an audio noise reduction method and related device, which can be applied to various scenes such as artificial intelligence and machine learning. By obtaining a target audio signal to be processed, a target sound type and a target noise type in the target audio signal are determined, a target noise reduction model corresponding to the target noise type is determined from N noise reduction models, the N noise reduction models are respectively used for targeted noise reduction on N noise types, noise in the target audio signal is processed by the target noise reduction model according to a noise reduction requirement indicated by the target sound type, and a noise reduction result corresponding to the target audio signal is obtained. According to the target sound type, the noise reduction requirement is determined, and then the noise in the target audio signal is reduced by the target noise reduction model according to the requirement, so that the influence on the target sound type corresponding to the effective sound is avoided, thereby realizing the targeted noise reduction on the audio signals of different target sound types and different target noise types.

Description

Technical Field

[0001] This application relates to the field of audio processing, and in particular to an audio noise reduction method and related apparatus. Background Technology

[0002] Noise is an interference signal for applications such as voice calls, human-computer voice interaction, and music recording. It directly interferes with the sound quality of effective sounds (such as speech and music) in the audio, which affects the effectiveness of voice calls and speech recognition.

[0003] Therefore, it is necessary to perform noise reduction processing on the collected audio to reduce the impact of noise in the audio, improve the speech signal-to-noise ratio, and enhance the clarity and intelligibility of the sound. Related technologies often employ a unified noise reduction method for audio denoising.

[0004] However, the requirements and methods for noise reduction vary depending on the audio application scenario, and it is difficult for a single noise reduction method to meet the noise reduction requirements in all different audio application scenarios. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides the following technical solution:

[0006] On one hand, embodiments of this application provide an audio noise reduction method, the method comprising:

[0007] Acquire the target audio signal to be processed;

[0008] Determine the target sound type corresponding to the valid sound and the target noise type corresponding to the noise in the target audio signal;

[0009] From N noise reduction models, determine the target noise reduction model corresponding to the target noise type. Among the N noise reduction models, there is a one-to-one correspondence between the N noise reduction models and the N noise types, where N>1.

[0010] Based on the noise reduction requirements indicated by the target sound type, the noise in the target audio signal is processed by the target noise reduction model to obtain the noise reduction result corresponding to the target audio signal.

[0011] On the other hand, embodiments of this application provide an audio noise reduction device, the device comprising:

[0012] The acquisition module is used to acquire the target audio signal to be processed.

[0013] The type determination module is used to determine the target sound type corresponding to the valid sound in the target audio signal and the target noise type corresponding to the noise.

[0014] The model determination module is used to determine the target noise reduction model corresponding to the target noise type from N noise reduction models. In the N noise reduction models, there is a one-to-one correspondence between the N noise reduction models and the N noise types, where N>1.

[0015] The processing module is used to perform noise reduction processing on the noise in the target audio signal according to the noise reduction requirements indicated by the target sound type, and to obtain the noise reduction result corresponding to the target audio signal.

[0016] In another aspect, embodiments of this application provide a computer device, the computer device including a processor and a memory:

[0017] The memory is used to store program code and transmit the program code to the processor;

[0018] The processor is used to execute the audio noise reduction method described above according to the instructions in the program code.

[0019] In another aspect, embodiments of this application provide a computer-readable storage medium for storing a computer program for performing the audio noise reduction method described above.

[0020] In another aspect, embodiments of this application provide a computer program product including instructions that, when run on a computer, cause the computer to perform the audio noise reduction method described above.

[0021] As can be seen from the above technical solution, this application provides an audio noise reduction method. It acquires the target audio signal to be processed, and then determines the target sound type corresponding to the effective sound and the target noise type corresponding to the noise in the target audio signal. Since the target sound type corresponding to the effective sound in different target audio signals may differ in different application scenarios, and the target noise type in the target audio signal may also differ, the application scenario of the target audio signal can be determined by the target sound type and target noise type, thereby achieving targeted noise reduction. A target noise reduction model corresponding to the target noise type is determined from N noise reduction models, where N is greater than 1, and the N noise reduction models correspond one-to-one with N noise types. Each of the N noise reduction models is used to perform targeted noise reduction on one of the N noise types. Based on the noise reduction requirements indicated by the target sound type, the noise in the target audio signal is processed using the target noise reduction model to obtain the noise reduction result corresponding to the target audio signal. Since the effective sounds in the target audio signal correspond to different target sound types, they may correspond to different noise reduction requirements. The noise reduction requirements can be determined based on the target sound type corresponding to the effective sounds. Then, based on these requirements, the noise in the target audio signal can be reduced using a target noise reduction model to avoid affecting the target sound type corresponding to the effective sounds. This achieves targeted noise reduction for audio signals with different target sound types and different target noise types. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram illustrating a scenario where an audio noise reduction method is provided in an embodiment of this application.

[0024] Figure 2 This is a flowchart illustrating an audio noise reduction method provided in an embodiment of this application;

[0025] Figure 3 This application provides an illustration of an application scenario for an audio noise reduction method.

[0026] Figure 4 This application provides a schematic diagram of the structure of a deep neural network model for determining the probability parameters of the target noise type and the probability parameters of the target sound type, according to an embodiment of the present application.

[0027] Figure 5 This application provides a schematic diagram of the structure of a comprehensive noise reduction neural network.

[0028] Figure 6 This application provides a schematic diagram of the structure of an audio noise reduction device according to an embodiment of the present application;

[0029] Figure 7 A structural diagram of a terminal device provided in an embodiment of this application;

[0030] Figure 8 This is a structural diagram of a server provided in an embodiment of this application. Detailed Implementation

[0031] The embodiments of this application will now be described with reference to the accompanying drawings.

[0032] Noise, also known as ambient noise, refers to sound that interferes with the acquisition of the desired sound; it is unwanted sound. Examples include horn noise, car engine noise, air conditioner noise, and electrical noise during voice calls. Similarly, talking sounds, door opening sounds, and air conditioner noise while listening to music are also considered noise.

[0033] For applications such as voice calls, human-computer voice interaction, and music recording, noise is an interference signal that directly interferes with the sound quality of effective sounds (such as speech and music) in the audio, affecting the quality of voice calls and the effectiveness of speech recognition.

[0034] Therefore, audio noise reduction is necessary to improve the signal-to-noise ratio of speech and enhance its clarity and intelligibility. While many related technologies employ standardized noise reduction methods, the requirements and methods for noise reduction vary depending on the specific audio application scenario. Furthermore, different types of noise require different noise reduction techniques.

[0035] For example, in voice calls, since speech is sparse in both the time and frequency domains, the noise reduction requirement is to suppress noise as much as possible to avoid interference with the call. In contrast, in music listening scenarios, music is non-sparse in both the time and frequency domains, with a wider signal spectrum coverage and a higher probability of overlap with noise. Therefore, the noise reduction requirement in music listening scenarios is to perform appropriate noise reduction without compromising the sound quality. Furthermore, for known noise, its amplitude can be determined, and noise reduction can be achieved by superimposing an inverse amplitude signal. However, a uniform noise reduction method only achieves good results when the scenarios are well-matched.

[0036] In view of this, this application provides an audio noise reduction method that can achieve noise reduction in various audio application scenarios and achieve targeted noise reduction for audio signals with different target sound types and different target noise types.

[0037] The audio noise reduction method provided in this application can be implemented using a computer device, which can be a terminal device or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal devices include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, and in-vehicle terminals. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this connection.

[0038] It is understood that in the specific embodiments of this application, the audio signals used may involve user information and other related data, such as the user's spoken voice and sung music. When the above embodiments of this application are applied to specific products or technologies, each one requires separate user permission or consent, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0039] This application relates to Artificial Intelligence (AI), which is the theory, method, technology, and application system for simulating, extending, and expanding human intelligence using digital computers or machines controlled by digital computers, perceiving the environment, acquiring knowledge, and using that knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. Artificial intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0040] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0041] The solutions provided in this application's embodiments relate to machine learning techniques in artificial intelligence, such as using machine learning to determine the target sound type corresponding to valid sound and the target noise type corresponding to noise in a target audio signal, wherein:

[0042] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0043] Figure 1 The illustration shows a scenario diagram of an audio noise reduction method provided in an embodiment of this application, wherein server 100 is used as an example of the aforementioned computer device for explanation.

[0044] Specifically, the server can obtain the target audio signal to be processed from the terminal, such as the target audio signal A collected in a voice call scenario, which includes the user's voice and noise during the voice call.

[0045] The server determines the target sound type and target noise type in the target audio signal. The target sound type refers to the sound type corresponding to the valid sound in the target audio signal, and the target noise type refers to the sound type corresponding to the noise in the target audio signal. Since the target sound type corresponding to the valid sound in different target audio signals may differ in different application scenarios, and the target noise type may also differ, the target sound type and target noise type can be used to determine the application scenario of the target audio signal, define the noise reduction requirements, and thus achieve targeted noise reduction for the target noise type.

[0046] For example, the target sound type corresponding to the effective sound in the target audio signal A is speech, and the target noise type corresponding to the noise is horn sound, car engine sound, air conditioner sound, and electrical sound.

[0047] Based on the determined target noise type, the server selects the target noise reduction model corresponding to the target noise type from N=10 noise reduction models. In this embodiment, the 10 noise reduction models are: horn noise reduction model, car engine noise reduction model, air conditioner noise reduction model, door opening noise reduction model, keyboard noise reduction model, printer noise reduction model, typing noise reduction model, car brake noise reduction model, mouse noise reduction model, and electrical noise reduction model. The noise types corresponding to the 10 noise reduction models are horn noise, car engine noise, air conditioner noise, door opening noise, keyboard noise, printer noise, typing noise, car brake noise, mouse noise, and electrical noise, respectively. Since the target noise types are horn noise, car engine noise, air conditioner noise, and electrical noise, the horn noise reduction model, car engine noise reduction model, air conditioner noise reduction model, and electrical noise reduction model corresponding to the target noise type are selected from the 10 noise reduction models.

[0048] Based on the noise reduction requirements indicated by the target sound type, the server uses the corresponding target noise reduction model to process the noise in the target audio signal, obtaining the corresponding noise reduction result. Different target sound types correspond to different noise reduction requirements. For example, when the target sound type is speech, the noise reduction requirement is to significantly improve the signal-to-noise ratio (SNR) after noise reduction, thereby improving the intelligibility and clarity of the speech. When the target sound type is music, the requirement is to suppress noise without compromising the music signal. Therefore, for target audio signal A, based on the target sound type being speech, the noise reduction requirement is determined to be improving the SNR. Targeted noise reduction is then performed using a horn noise reduction model for horn sounds in target audio signal A, a car engine noise reduction model for car engine sounds in target audio signal A, an air conditioner noise reduction model for air conditioner sounds in target audio signal A, and a current noise reduction model for current sounds in target audio signal A. Finally, a noise reduction result with a high SNR is obtained after denoising each type of noise. The server returns the noise reduction result with a high signal-to-noise ratio to the terminal, thus achieving noise reduction processing of the target audio signal.

[0049] Thus, for any target audio signal in any application scenario, the server can determine the target sound type corresponding to the effective sound and the target noise type corresponding to the noise in the target audio signal, determine the target noise reduction model corresponding to the target noise type, and perform noise reduction processing on the noise in the target audio signal according to the noise reduction requirements indicated by the target sound type, thereby obtaining the corresponding noise reduction result. On the one hand, noise reduction processing using the target noise model corresponding to the target noise type can achieve targeted noise reduction. On the other hand, noise reduction processing according to the noise reduction requirements indicated by the target sound type can fully consider the characteristics of different target sound types and obtain noise reduction results that meet the noise reduction requirements of the target sound type.

[0050] Figure 2 This is a flowchart illustrating an audio noise reduction method provided in an embodiment of this application. In this embodiment, server 100 is used as the aforementioned computer device for description. The method includes the following steps:

[0051] S202: The server obtains the target audio signal to be processed.

[0052] The target audio signal refers to an audio signal that includes both valid sound and noise. The target audio signal can be a target audio signal collected in any scenario, such as a target audio signal collected in a phone call scenario or a target audio signal collected in a music scenario.

[0053] A server can obtain the target audio signal to be processed through various means. For example, the server can obtain the target audio signal to be processed sent by the terminal device, or it can obtain the target audio signal from other servers.

[0054] To ensure noise reduction efficiency, a length limit can be set for the audio signal to be denoised in a single operation. When the audio duration of the noise frequency to be denoised is long, the server can collect the target audio signal from the noise frequency in segments. The length of the target audio signal to be processed can be fixed or variable. For example, the server can periodically collect the target audio signal from the noise frequency to be denoised based on a fixed sampling audio frame interval. Considering the continuity of sound, the server can also acquire the target audio signal to be processed in units of pauses in audio frames, sequentially determining the target sound type and target noise type corresponding to each segment of the target audio signal.

[0055] By periodically acquiring the target audio signal from the noise-to-be-reduced frequency and performing noise reduction processing, the noise reduction efficiency and response speed of the noise-to-be-reduced frequency can be improved. The noise-reduced portion of the noise-to-be-reduced frequency can be played and used, while the un-noise-reduced portion is simultaneously noise-reduced during playback and use. S204: The server determines the target sound type corresponding to the valid sound in the target audio signal and the target noise type corresponding to the noise.

[0056] The target sound type refers to the sound type corresponding to the valid sound in the target audio signal, that is, the sound type that needs to be retained in the target audio signal. The target noise type refers to the sound type corresponding to the noise in the target audio signal, that is, the sound type that needs to be removed from the target audio signal.

[0057] Different target sound types correspond to different characteristics. For example, target sound types can include speech and music. Speech signals are sparsely distributed in both the time and frequency domains, while music signals have a wider signal spectrum coverage and a non-sparse distribution in both the time and frequency domains. Due to these different characteristics, speech and music signals can correspond to different noise reduction requirements. Therefore, by identifying the target sound type of the target audio signal among multiple sound types, the noise reduction requirements for the target audio signal can be accurately determined, thereby improving noise reduction accuracy.

[0058] Noise type refers to the classification of interfering sounds. Different types of noise have different characteristics. Noise can be classified based on its state, such as steady-state noise and non-steady-state noise. Steady-state noise refers to continuous noise with intensity fluctuations within 5 dB, or impulse noise with a repetition frequency greater than 10 Hz. Steady-state noise is characterized by its continuous existence and its loudness, frequency distribution, and other characteristics not changing or changing slowly over time. Non-steady-state noise refers to noise whose characteristics change over time. Furthermore, non-steady-state noise is further divided into continuous non-steady-state noise and transient noise. Due to the different characteristics of different noise states, noise reduction methods corresponding to the noise state can be adopted. For example, based on the repetitive nature of steady-state noise, a small segment of noise can be sampled to obtain its spectral energy, and then subtracted from the total noise to achieve noise reduction. Another example is that the location of non-steady-state noise can be determined based on spectral differences, and then processed accordingly.

[0059] Furthermore, some noises operate in the low-to-mid frequency range, some in the low frequency range, and some in the high frequency range. Noises at the same frequency share similar characteristics and can be treated with the same noise reduction methods. Therefore, noise can also be classified based on its frequency. Similarly, noise can be classified into various types based on its loudness, amplitude, and source. For example, noise can be further subdivided into types such as electrical noise, fan noise, air conditioner noise, and car engine noise based on its source.

[0060] Since noise with the same sound state has the same noise characteristics, it can be targeted by the same noise reduction method and achieve similar noise reduction effects.

[0061] Noise in the same sound scene can exhibit similar characteristics across different regions. For instance, the noise generated in airports in different regions tends to be quite similar. In other words, while the noise's sound characteristics may vary within a single sound scene, their overall features are similar. Therefore, audio signals collected from the same sound scene in different regions, such as airports, can be denoised using the same noise reduction methods, achieving similar noise reduction effects. Thus, noise types can be categorized based on sound scene, such as... Figure 3 As shown, for the four sound scenarios of streets (Part A), airports (Part B), offices (Part C), and subway stations (Part D), the noise types can be divided into street noise, airport noise, office noise, and subway station noise. Noise reduction models corresponding to these sound scenarios are trained for subsequent targeted noise reduction.

[0062] For a target audio signal to be processed, the server can determine the noise type based on the noise's sound state, or it can determine the noise type based on the sound scene. For example, for a target audio signal to be processed, the server can determine that the noise is steady-state noise and airport noise, and then it can perform noise reduction for steady-state noise and airport noise respectively to achieve better noise reduction results.

[0063] When the noise type is classified based on the noise's sound state, the location of the noise can be determined according to its type, allowing for noise reduction at that specific location. When the noise type is classified based on the sound scene, typically only one noise reduction model corresponding to that scene needs to be called to reduce noise in that scene, thus improving noise reduction efficiency. Therefore, classifying noise based on its different characteristics allows for targeted noise reduction for noises with specific characteristics, achieving better noise reduction results.

[0064] For some application scenarios with more refined noise reduction requirements, noise reduction can be performed on the target audio signal in the application scenario by both the noise sound state and the sound scene, so as to achieve more comprehensive noise reduction from multiple dimensions and improve the noise reduction effect.

[0065] When determining the type of noise in a target audio file, a probability parameter can be used to quantify the similarity between the noise type in the target audio file and various other noise types. The probability parameter represents the probability that the noise in the target audio file is that particular type of noise.

[0066] Therefore, in one possible implementation, the target noise type can be determined in the following way:

[0067] S11: The server can determine the probability parameters corresponding to the noise and N noise types in the target audio signal.

[0068] S12: Determine the target noise type from N noise types based on probability parameters.

[0069] The server can obtain the spectral characteristics corresponding to various noise types, and then compare the spectral characteristics of the noise in the target audio signal with the spectral characteristics of the various noise types to obtain the similarity between the noise in the target audio signal and the various noise types. Based on the similarity with the various noise types, the server determines the probability parameter of the noise in the target audio signal as each noise type, and then determines the corresponding target noise type based on the probability parameter.

[0070] For example, there are N noise types, namely steady-state noise, continuous non-steady-state noise, and transient noise. Then, the similarity between the spectral characteristics of the noise in the target audio signal and these three noise types is determined to be 0.7, 0.2, and 0.6, respectively. Then, the probability parameters corresponding to the noise in the target audio signal and these three noise types are determined to be 0.7, 0.2, and 0.6, respectively. Finally, based on the probability parameters, the target noise type is determined to be steady-state noise and transient noise.

[0071] When N noise types are street noise, airport noise, office noise, and subway station noise, the server can determine the corresponding probability parameters by comparing the similarity of the spectral feature data of the noise in the target audio signal with the spectral feature data of the noise in the four scenarios. For example, the probability parameter is 0.6 for street noise, 0.5 for airport noise, 0.2 for office noise, and 0.8 for subway station noise, thus determining that the corresponding target noise type is subway station noise.

[0072] The server can also use a deep neural network model to determine the probability parameters corresponding to noise in the target audio type and N noise types, and then determine the target noise type from the N noise types based on the probability parameters. Specifically, the spectral feature data of the target audio signal can be input into a deep neural network for signal classification, and the output of the deep neural network model can be the probability parameters of the target audio signal being different noise types.

[0073] Typically, the acquired target audio signal is in the time domain, therefore it needs to be converted into a frequency domain signal using Fourier transform. Specifically, for target audio signals in the time domain, a Fourier transform must first be performed, for example, using discrete cosine transform (DCT) or fast Fourier transform (FFT) to obtain the spectral feature data of the target audio signal, which can then be used as input to a deep neural network model.

[0074] This deep neural network model can include convolutional (conv) layers, gated recurrent unit (GRU) layers, long short-term memory (LSTM) layers, fully connected (FC) layers, and related variant layers. For example, such as Figure 4 As shown, in this deep neural network, the frequency domain feature data of the target audio signal can be sequentially passed through three one-dimensional convolutional (conv1d) layers, two gated recurrent unit layers, a fully connected layer, and a normalized exponential function (softmax) to obtain the probability parameters of noise in the target audio signal under various noise types.

[0075] When the probability parameter of a certain type of noise corresponding to the target audio signal is large, it indicates that there is a high probability that the target audio signal contains that type of noise. Generally, the more prominent a certain type of noise is in the target audio signal, the larger its corresponding probability parameter. Therefore, the probability parameters can be used to determine the types of target noise that may be included in the target audio signal, and then targeted noise reduction can be performed using the corresponding target noise model. Furthermore, the probability parameters corresponding to N noise types can quantify the prominence of various types of noise in the target audio signal, and can also clearly reflect the differences in the impact of multiple target noise types on the effective sound in the target audio signal, so that subsequent noise reduction can be performed according to the probability parameters.

[0076] Furthermore, this deep neural network can also be used to classify the target sound type corresponding to the valid sound, obtaining the target sound type of the target audio signal, such as speech or music. For example, the deep neural network can output the probability that the target sound type corresponding to the valid sound in the target audio signal is speech or music, and then take the one with the higher probability as the target sound type corresponding to the valid sound.

[0077] In one possible implementation, the aforementioned S12: the server determines the target noise type from N noise types based on the probability parameter, which may include: determining the noise type with a probability parameter greater than the probability threshold value as the target noise type from the N noise types.

[0078] The probability threshold can be determined based on the current noise reduction accuracy and requirements. The probability threshold can be used to identify the types of noise that are more likely to exist in the target audio signal, or the types of noise that have a greater impact on the effective sound.

[0079] After obtaining the probability parameters corresponding to the noise and N noise types in the target audio signal through a deep neural network, M target noise types can be determined from the N noise types by setting probability thresholds. For example, the probability threshold can be 0.5. A probability parameter greater than 0.5 indicates that there is a greater than 50% chance that the audio signal contains that noise type. Therefore, noise types with probability parameters greater than the probability threshold (0.5) can be identified as target noise types. In this way, the server can identify noise types with probability parameters greater than the threshold as target noise types in the case of multiple noise types, thereby achieving the effect of accurately determining the noise type in terms of probability or degree of influence.

[0080] Alternatively, the server can set the value of M to determine the M target noise types with the highest probability parameters from N noise types based on probability parameters. For example, M can be 3 and N can be 10. The server can obtain the probability parameters of each of the 10 noise types for the target audio signal, and then select the 3 noise types with the highest probability parameters as the target noise types. As another example, M can be 1 and N can be 4, representing four noise types: street noise, airport noise, office noise, and subway station noise. The probability parameters for street noise are 0.6, airport noise is 0.5, office noise is 0.2, and subway station noise is 0.8, thus determining the corresponding target noise type as subway station noise.

[0081] S206: The server determines the target noise reduction model corresponding to the noise type from N noise reduction models.

[0082] This solution includes N noise reduction models, where N > 1. Each of the N noise reduction models corresponds one-to-one with one of the N noise types. For example, when the noise types include three categories: steady-state noise, continuous non-steady-state noise, and transient noise, this solution includes three noise reduction models: a steady-state noise reduction model, a continuous non-steady-state noise reduction model, and a transient noise reduction model. When the noise types include eight categories: electrical current noise, fan noise, air conditioner noise, car engine noise, background human voices, door closing noise, keyboard noise, and coughing noise, this solution includes eight noise reduction models: an electrical current noise reduction model, a fan noise reduction model, an air conditioner noise reduction model, a car engine noise reduction model, a background human voice noise reduction model, a door closing noise reduction model, a keyboard noise reduction model, and a coughing noise reduction model. When the noise types include street noise, airport noise, office noise, and subway station noise, this solution includes four noise reduction models: a street noise noise reduction model, an airport noise reduction model, an office noise reduction model, and a subway station noise reduction model.

[0083] S208: The server performs noise reduction processing on the target audio signal according to the noise reduction requirements indicated by the target sound type, and obtains the noise reduction result corresponding to the target audio signal.

[0084] Noise reduction requirements indicate the expected outcome of the noise reduction. Since the target sound type has different signal characteristics, the noise reduction requirements also differ. For example, for effective speech sounds, which are sparse in the time and frequency domains, the corresponding noise reduction requirement is to suppress noise as much as possible. For effective music sounds, which are not sparse in the time and frequency domains and may have significant overlap with noise, the corresponding noise reduction requirement is to perform appropriate noise reduction without damaging the effective sound as much as possible.

[0085] Therefore, in the process of targeted noise reduction of the target audio signal based on the identified target noise type, the effective sound in the target audio signal is also protected by noise reduction requirements, so as to avoid or reduce the substantial impact of noise reduction on the effective sound and stabilize the sound quality of the effective sound in the noise reduction result.

[0086] It can be seen that, according to the noise reduction requirement indicated by the target sound type, the noise in the target audio signal is denoised by the target noise reduction model to obtain the noise reduction result corresponding to the target audio signal. Since the target sound types corresponding to effective sounds in the target audio signal are different, different noise reduction requirements may be corresponded. The noise reduction requirement can be determined according to the target sound type corresponding to the effective sound, and then the noise in the target audio signal is denoised by the target noise reduction model according to the requirement, which avoids affecting the target sound type corresponding to the effective sound, thereby realizing targeted noise reduction for audio signals with different target sound types and different target noise types.

[0087] In some possible implementations, the foregoing step S208 includes: determining a corresponding noise reduction intensity according to the target sound type as the noise reduction requirement; performing noise reduction processing on the noise in the target audio signal through the target noise reduction model to obtain spectrum gain data; determining the noise reduction result corresponding to the target audio signal according to the noise reduction intensity and the spectrum gain data.

[0088] The noise reduction requirement can be represented by the noise reduction intensity. The noise reduction intensity refers to the processing intensity of noise reduction in the target audio signal. The greater the noise reduction intensity is, the greater the data loss of the spectrum where the noise is located will be. For example, when a higher noise reduction intensity is required, the spectrum gain data can be squared; when a lower noise reduction intensity is required, the square root operation can be performed on the spectrum gain data.

[0089] Due to the different characteristics of voice type and music type, the noise reduction intensity corresponding to the voice type is greater than that corresponding to the music type. When the target sound type is the voice type, the corresponding noise reduction requirement is to suppress noise as much as possible, avoid noise interference to voice and prevent noise from affecting the accuracy of voice recognition. When the target sound type is the music type, the corresponding noise reduction requirement is to perform appropriate noise reduction without damaging the music signal. Therefore, different noise reduction requirements can be determined based on different target sound types, so as to meet the noise reduction requirements in different environments.

[0090] Specifically, the server may determine the corresponding noise reduction intensity according to the target sound type as the noise reduction requirement, perform noise reduction processing on the noise in the target audio signal through the target noise reduction model corresponding to the noise type to obtain spectrum gain data, and then determine the noise reduction result corresponding to the target audio signal according to the noise reduction intensity and the spectrum gain data. When the target sound type is voice, the corresponding noise reduction intensity is relatively high, so the spectrum gain data can be squared, and then the noise reduction result corresponding to the target audio signal is determined. When the target sound type is music, the corresponding noise reduction intensity is relatively low, so the square root operation is performed on the spectrum gain data to determine the noise reduction result corresponding to the target audio signal.

[0091] In this way, the server can determine the noise reduction requirements based on the target sound type corresponding to the effective sound, and then adjust the spectral gain data obtained by the noise reduction model by using the noise reduction intensity as the noise reduction requirement, thereby obtaining noise reduction requirements that meet different types of target sounds.

[0092] When the number of target noise types M ≥ 2, that is, when it is determined that the target audio signal has multiple noise types, S208 may include:

[0093] S2081: Using M target noise reduction models corresponding to M target noise types, noise reduction processing is performed on the noise in the target audio signal to obtain M spectral gain data corresponding to the M target noise reduction models.

[0094] Among them, the spectral gain data represents the intermediate noise reduction result after the target audio signal is removed from the noise of the corresponding target noise type.

[0095] Specifically, the M spectral gain data are the M spectral gain data obtained after each of the M denoising models has processed the corresponding noise. For example, the target audio signal includes valid sound, horn sound, car engine sound, air conditioner sound, and electrical noise. After the horn sound denoising model, the corresponding spectral gain data includes valid sound, car engine sound, air conditioner sound, and electrical noise. After the car engine sound denoising model, the corresponding spectral gain data includes valid sound, horn sound, air conditioner sound, and electrical noise. After the air conditioner sound denoising model, the corresponding spectral gain data includes valid sound, horn sound, car engine sound, and electrical noise. After the electrical noise denoising model, the corresponding spectral gain data includes valid sound, horn sound, car engine sound, and air conditioner sound.

[0096] The server determines the noise reduction result for the target audio signal based on M spectral gain data, the probability parameters corresponding to M target noise types, and the noise reduction requirements indicated by the target sound type. The M spectral gain data represent the noise reduction result of the target audio signal after removing noise of that specific type.

[0097] S2082: The server can process M spectral gain data according to the probability parameters corresponding to M target noise types and the noise reduction requirements indicated by the target sound type to obtain the corresponding noise reduction results.

[0098] Since the probability parameters corresponding to the M target noise types may be different, for the M spectral gain data, for example, the spectral gain data corresponding to the target noise type with a larger probability parameter can be amplified, and the spectral gain data corresponding to the target noise type with a smaller probability parameter can be reduced, so as to perform corresponding noise reduction according to the obviousness of different noise types in the target audio signal.

[0099] For example, when N=3, the three target noise types are horn sound, air conditioner sound and current sound. The probability parameters of horn sound, air conditioner sound and current sound in the target audio signal are 0.9, 0.6 and 0.4, respectively, and the threshold value is 0.5. Thus, M=2 target noise types are determined, namely horn sound and air conditioner sound.

[0100] As mentioned earlier, probability parameters can reflect the likelihood of the corresponding noise type in the target audio signal, or the degree of its influence on the effective sound. Moreover, when there are multiple target noise types, they can also reflect the differences in their influence on the effective sound in the target audio signal.

[0101] Based on the two target noise types identified above, when determining the corresponding noise reduction results, the influence of the spectral gain data corresponding to speaker noise on the noise reduction results can be increased, while the influence of the spectral gain data corresponding to air conditioner noise on the noise reduction results can be decreased. This can increase the proportion of high-probability parameter spectral gain data in the noise reduction results, increase the noise reduction accuracy, and reduce the impact of significant noise on the effective sound.

[0102] like Figure 5 The diagram shows the structure of a comprehensive noise reduction neural network. The target audio signal undergoes a Fast Fourier Transform (FFT) to obtain its corresponding frequency domain data. This frequency domain data is then processed to obtain the power spectrum of the target audio signal. Based on the probability parameters corresponding to the noise and N noise types in the target audio signal, probability judgments are made to determine M target noise types with probability parameters greater than a probability threshold, corresponding to M target noise reduction models.

[0103] exist Figure 5 In this process, the probability judgment result can be used to indicate the opening and closing of the switches of N noise reduction models. When it is determined that the noise type corresponding to the noise reduction model is the target noise type, that is, it belongs to the target noise reduction model, the switch of the target noise reduction model will be closed, and the power spectrum of the target audio signal will be input into the corresponding target noise reduction model. Otherwise, the switch of the noise reduction model will be in the off state, and the power spectrum of the target audio signal will not be input into the noise reduction model. After the power spectrum of the target audio signal is processed by M target noise reduction models, M spectral gain data corresponding to each of the M target noise reduction models are obtained.

[0104] To improve the accuracy of determining the noise reduction result using M spectral gain data, in one possible implementation, S2082 may include:

[0105] Based on the M spectral gain data, the probability parameters corresponding to the M target noise types, and the noise reduction requirements, the final spectral gain data is determined through a fully connected network. The probability parameters are used to identify the degree of influence of the corresponding spectral gain data in the final spectral gain data.

[0106] Based on the final spectral gain data, the noise reduction result corresponding to the target audio signal is determined.

[0107] In a fully connected network, probability parameters are used to identify the degree of influence of the spectral gain data corresponding to a certain type of noise, while noise reduction requirements are used to identify the target sound type in the target audio signal. The fully connected network is used to fuse M spectral gain data, the probability parameters corresponding to M target noise types, and the noise reduction requirements to obtain the final spectral gain data.

[0108] Through fully connected networks (such as those mentioned above) Figure 5 (As shown) this can enhance the fusion of M spectral gain data. Through data fusion, each spectral gain data can learn information corresponding to other target noise types. Furthermore, the probability parameters corresponding to each of the M target noise types can guide the proportion of each spectral gain data in the final spectral gain data during fusion, making the proportion of each spectral gain data in the final spectral gain data more reasonable and more consistent with the actual impact of noise on the effective sound, or the target audio signal. In addition, the characteristics of the effective sound in the target audio signal reflected in the noise reduction requirements result in the final spectral gain data used to determine the noise reduction result having richer information than the M spectral gain data, effectively improving the accuracy of the determined noise reduction result.

[0109] The server can multiply the final spectral gain data by the power spectrum of the target audio signal, and then synthesize the phase of the target audio signal through inverse Fourier transform (IFT) to obtain the corresponding time-domain signal, which is the final noise reduction result.

[0110] In one possible implementation of S202 mentioned above, a periodic sampling method for the noise frequency to be reduced is described. Based on this, this application can also achieve noise reduction processing of the noise frequency to be reduced according to the noise reduction result corresponding to each target audio signal of the noise frequency to be reduced.

[0111] Therefore, in one possible implementation, the method further includes: the server completing the noise reduction processing of the noise frequency to be reduced based on the noise reduction results corresponding to the target audio signals collected in each cycle.

[0112] For example, when periodically acquiring target audio signals from noise frequencies to be reduced, the server can periodically acquire target audio signals from noise frequencies to be reduced based on the sampling audio frame interval, and then apply the audio noise reduction method of this scheme to the target audio signals in units of sampling audio frame intervals.

[0113] In this way, when the effective sound and noise in the target audio signal change, it can be quickly identified, and the corresponding noise reduction model and noise reduction requirements can be changed to perform corresponding noise reduction on the changed target audio signal.

[0114] The sampling audio frame interval can be set according to the required sensitivity. When a higher sensitivity is needed, a smaller sampling audio frame interval can be set to improve the sensitivity of the method. After obtaining the noise reduction result corresponding to the target audio signal in each cycle, the server completes the noise reduction processing of the noise frequency to be denoised based on the noise reduction result corresponding to the target audio signal collected in each cycle.

[0115] In one possible implementation, for a segment of audio signal to be noise-reduced, the server periodically acquires the target audio signal within one cycle using audio frame intervals. Then, it determines the target sound type corresponding to the effective sound and the target noise type corresponding to the noise within each cycle. From N noise reduction models, it selects the target noise reduction model corresponding to the target noise type. Based on the noise reduction requirements indicated by the target sound type, it performs noise reduction processing on the target audio signal using the target noise reduction model, obtaining the noise reduction result for each cycle of the target audio signal. Finally, it combines the noise reduction results from each cycle to complete the noise reduction processing of the audio signal to be noise-reduced. In this way, the target sound type and target noise type are identified for each cycle of the target audio signal. When the effective sound and noise in the target audio signal change, the transformation can be quickly identified, and the corresponding noise reduction model and noise reduction requirements can be changed, thereby improving the sensitivity of the identification.

[0116] In another possible implementation, for a segment of audio to be noise-reduced, the server can identify the target audio signal in the first cycle to obtain the target sound type corresponding to the effective sound and the target noise type corresponding to the noise. Then, it determines the target noise reduction model corresponding to the target noise type from N noise reduction models. Based on the noise reduction requirements indicated by the target sound type in the first cycle, it uses the target noise reduction model corresponding to the first cycle to perform noise reduction processing on the target audio signal in each cycle of the entire audio segment to be noise-reduced, obtaining the noise reduction result corresponding to the target audio signal in each cycle. Finally, it combines the noise reduction results in each cycle to complete the noise reduction processing of the audio segment to be noise-reduced. In this way, considering the stability of the user's environment, it is not necessary to make judgments for each cycle, thereby improving the noise reduction efficiency.

[0117] To make the technical solution of this application clearer and easier to understand, the embodiments of this application also address... Figure 3 The specific scenarios in the text illustrate audio noise reduction methods with examples.

[0118] like Figure 3 As shown in (A), the scene is a street. The server acquires the audio collected by the microphone as the target audio signal to be processed, determines the target sound type corresponding to the effective sound as speech, and the target noise types corresponding to the noise as tire friction sound, car horn sound, car brake sound, and car motor sound, etc. The server selects the tire friction sound noise reduction model, car horn sound noise reduction model, car brake sound noise reduction model, and car motor sound noise reduction model as the target noise reduction models from N noise reduction models. Then, according to the noise reduction requirements indicated by the speech, the server performs noise reduction processing on the above-mentioned noises in the target audio signal using the tire friction sound noise reduction model, car horn sound noise reduction model, car brake sound noise reduction model, and car motor sound noise reduction model to obtain the corresponding noise reduction results.

[0119] like Figure 3 As shown in (B), the scenario is an airport. The server obtains the audio sent by the other server as the target audio signal to be processed, determines that the target sound type corresponding to the valid sound is speech, and the target noise type corresponding to the noise is airport noise. The server selects the airport noise denoising model as the target denoising model from N denoising models, and then performs denoising processing on the airport noise in the target audio signal through the airport noise denoising model according to the denoising requirements indicated by the speech, to obtain the corresponding denoising result.

[0120] like Figure 3 As shown in (C), the scenario is an office. The server acquires the music played by the speakers and the office noise as the target audio signal, determines the target sound type corresponding to the effective sound as music, and the target noise types corresponding to the noise as keyboard sounds, mouse sounds, conversations, and desk and chair friction sounds. The server selects the corresponding noise reduction models from N noise reduction models as the keyboard sound noise reduction model, mouse sound noise reduction model, conversation noise reduction model, and desk and chair friction sound noise reduction model. Then, according to the noise reduction requirements indicated by the music, the server performs noise reduction processing on the keyboard sounds, mouse sounds, conversations, and desk and chair friction sounds in the target audio signal using the above noise reduction models to obtain the corresponding noise reduction results.

[0121] like Figure 3As shown in (D), the scenario is a subway station. The server acquires the music played by the speakers and the subway station noise as the target audio signals, determines the target sound type corresponding to the effective sound as music, and the target noise type corresponding to the noise as subway station noise. The server selects the subway station noise noise reduction model from N noise reduction models, and then, according to the noise reduction requirements indicated by the music, performs noise reduction processing on the subway station noise in the target audio signal using the subway station noise noise reduction model to obtain the corresponding noise reduction result.

[0122] In the foregoing Figures 1-5 Based on the corresponding embodiments, Figure 6 This is a schematic diagram of an audio noise reduction device 600, which includes: an acquisition module 602, a type determination module 604, a model determination module 606, and a processing module 608.

[0123] Acquisition module 602 is used to acquire the target audio signal to be processed;

[0124] The type determination module 604 is used to determine the target sound type corresponding to the valid sound in the target audio signal and the target noise type corresponding to the noise.

[0125] The model determination module 606 is used to determine the target noise reduction model corresponding to the target noise type from N noise reduction models, wherein the N noise reduction models correspond one-to-one with the N noise types, and N>1;

[0126] The processing module 608 is used to perform noise reduction processing on the noise in the target audio signal according to the noise reduction requirements indicated by the target sound type, and to obtain the noise reduction result corresponding to the target audio signal.

[0127] In one possible implementation, the processing module 608 is specifically used for:

[0128] The corresponding noise reduction intensity is determined based on the target sound type as the noise reduction requirement;

[0129] The noise in the target audio signal is reduced using the target noise reduction model to obtain spectral gain data.

[0130] Based on the noise reduction intensity and the spectral gain data, the noise reduction result corresponding to the target audio signal is determined.

[0131] In one possible implementation, the target sound type includes a speech type or a music type, and the noise reduction level corresponding to the speech type is greater than the noise reduction level corresponding to the music type.

[0132] In one possible implementation, the type determination module 604 is specifically used for:

[0133] Determine the probability parameters corresponding to the noise and the N noise types in the target audio signal, respectively;

[0134] Based on the probability parameters, the target noise type is determined from the N noise types, where the number of target noise types is M, and M≤N.

[0135] In one possible implementation, the type determination module 604 is specifically used for:

[0136] From the N noise types, the noise type whose probability parameter is greater than the probability threshold value is determined as the target noise type.

[0137] In one possible implementation, if M ≥ 2, the processing module 608 is specifically used for:

[0138] The noise in the target audio signal is denoised using M target noise reduction models corresponding to the target noise types, resulting in M ​​spectral gain data, each of which corresponds one-to-one with one of the M target noise reduction models.

[0139] Based on the noise reduction requirements indicated by the target sound type, the noise reduction result corresponding to the target audio signal is determined by using the M spectral gain data and the probability parameters corresponding to the M target noise types.

[0140] In one possible implementation, the processing module 608 is specifically used for:

[0141] Based on the M spectral gain data, the probability parameters corresponding to the M target noise types, and the noise reduction requirements, the final spectral gain data is determined through a fully connected network. The probability parameters are used to identify the degree of influence of the corresponding spectral gain data in the final spectral gain data.

[0142] Based on the final spectral gain data, the noise reduction result corresponding to the target audio signal is determined.

[0143] In one possible implementation, the noise type includes noise types classified based on noise sound state, and / or includes noise types classified based on sound scene.

[0144] In one possible implementation, the acquisition module 602 is specifically used for:

[0145] The target audio signal is periodically acquired from the noise frequency to be reduced based on the sampling audio frame interval;

[0146] The device also includes an integration module for:

[0147] Based on the noise reduction results of the target audio signal collected in each cycle of the noise frequency to be reduced, the noise reduction processing of the noise frequency to be reduced is completed.

[0148] Therefore, by acquiring the target audio signal to be processed, the target sound type corresponding to the effective sound and the target noise type corresponding to the noise in the target audio signal can be determined. Since the target sound type corresponding to the effective sound in different target audio signals may differ in different application scenarios, and the target noise type may also differ, the application scenario of the target audio signal can be determined by the target sound type and target noise type, thereby achieving targeted noise reduction. A target noise reduction model corresponding to the target noise type is determined from N noise reduction models, where N is greater than 1, and each of the N noise reduction models corresponds one-to-one with one of the N noise types. Each of the N noise reduction models is used to perform targeted noise reduction on one of the N noise types. Based on the noise reduction requirements indicated by the target sound type, the noise in the target audio signal is processed using the target noise reduction model to obtain the corresponding noise reduction result for the target audio signal. Since the effective sounds in the target audio signal correspond to different target sound types, they may correspond to different noise reduction requirements. The noise reduction requirements can be determined based on the target sound type corresponding to the effective sounds. Then, based on these requirements, the noise in the target audio signal can be reduced using a target noise reduction model to avoid affecting the target sound type corresponding to the effective sounds. This achieves targeted noise reduction for audio signals with different target sound types and different target noise types.

[0149] This application also provides a computer device, which is the computer device described above and may include a terminal device or a server. The computer device will now be described in conjunction with the accompanying drawings.

[0150] If the computer device is a terminal device, please refer to Figure 7 As shown, this application provides a terminal device, taking a mobile phone as an example:

[0151] Figure 7 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 7 The mobile phone includes components such as a radio frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a Wi-Fi module 770, a processor 780, and a power supply 790. Those skilled in the art will understand that... Figure 7The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0152] The following is combined Figure 7 A detailed introduction to each component of a mobile phone:

[0153] The RF circuit 710 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 780; in addition, it transmits uplink data to the base station.

[0154] The memory 720 can be used to store software programs and modules. The processor 780 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0155] The input unit 730 can be used to receive input numeric or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732.

[0156] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 740 may include a display panel 741.

[0157] The mobile phone may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors.

[0158] Audio circuitry 760, speaker 761, and microphone 762 provide an audio interface between the user and the mobile phone.

[0159] WiFi is a short-range wireless transmission technology. Through the WiFi module 770, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access.

[0160] The processor 780 is the control center of the mobile phone. It connects various parts of the mobile phone through various interfaces and lines. It performs various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 720 and calling data stored in the memory 720.

[0161] The phone also includes a power supply 790 (such as a battery) that powers the various components.

[0162] In this embodiment, the processor 780 included in the terminal device also has the following functions:

[0163] Acquire the target audio signal to be processed;

[0164] Determine the target sound type corresponding to the valid sound and the target noise type corresponding to the noise in the target audio signal;

[0165] From N noise reduction models, determine the target noise reduction model corresponding to the target noise type. Among the N noise reduction models, there is a one-to-one correspondence between the N noise reduction models and the N noise types, where N>1.

[0166] Based on the noise reduction requirements indicated by the target sound type, the noise in the target audio signal is processed by the target noise reduction model to obtain the noise reduction result corresponding to the target audio signal.

[0167] If the computer device is a server, this application embodiment also provides a server; please refer to [link to relevant documentation]. Figure 8 As shown, Figure 8 This is a structural diagram of a server 800 provided in an embodiment of this application. The server 800 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 822 (e.g., one or more processors) and a memory 832, and one or more storage media 830 (e.g., one or more mass storage devices) for storing application programs 842 or data 844. The memory 832 and storage media 830 can be temporary or persistent storage. The program stored in the storage media 830 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 822 may be configured to communicate with the storage media 830 and execute the series of instruction operations in the storage media 830 on the server 800.

[0168] Server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input / output interfaces 858, and / or one or more operating systems 841, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0169] The steps performed by the server in the above embodiments can be based on Figure 8 The server structure shown.

[0170] In addition, this application embodiment also provides a storage medium for storing a computer program for executing the method provided in the above embodiment.

[0171] This application also provides a computer program product including instructions that, when run on a computer, cause the computer to perform the methods provided in the above embodiments.

[0172] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0173] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0174] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Moreover, based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An audio noise reduction method, characterized in that, The method includes: Acquire the target audio signal to be processed; Determine the target sound type corresponding to the valid sound in the target audio signal and the target noise type corresponding to the noise, wherein the target sound type includes speech type and music type; The target noise type is determined as follows: the probability parameters corresponding to the noise and N noise types in the target audio signal are determined respectively. The probability parameters are used to indicate the probability of the corresponding noise type in the target audio signal and the degree of influence on the effective sound. Based on the probability parameters, the target noise type is determined from the N noise types. The number of target noise types is M, 2≤M≤N. From N noise reduction models, a target noise reduction model corresponding to the target noise type is determined. Among the N noise reduction models, each noise reduction model corresponds one-to-one with one of the N noise types, where N>1. The N noise reduction models include steady-state noise reduction models, continuous non-steady-state noise reduction models, and transient noise reduction models. The N noise types include steady-state noise, continuous non-steady-state noise, and transient noise. Based on the noise reduction requirements indicated by the target sound type, the noise in the target audio signal is reduced using the target noise reduction model to obtain the noise reduction result corresponding to the target audio signal, specifically including: The noise in the target audio signal is denoised using M target noise reduction models corresponding to the target noise types, resulting in M ​​spectral gain data, each of which corresponds one-to-one with one of the M target noise reduction models. The M spectral gain data, the probability parameters corresponding to the M target noise types, and the noise reduction requirements are fused by a fully connected network to obtain the final spectral gain data. The probability parameters are used to identify the degree of influence of the corresponding spectral gain data in the final spectral gain data. The spectral gain data corresponding to the target noise type with a larger probability parameter is amplified, and the spectral gain data corresponding to the target noise type with a smaller probability parameter is reduced. The noise reduction requirement is used to protect the effective sound in the target audio signal. The protection is used to avoid the noise reduction process from affecting the effective sound. The noise reduction requirement includes the noise reduction strength corresponding to the target sound type being speech or music. The noise reduction strength corresponding to speech is greater than the noise reduction strength corresponding to music. If the target sound type is speech, the spectral gain data is squared. If the target sound type is music, the spectral gain data is squared. Based on the final spectral gain data, the noise reduction result corresponding to the target audio signal is determined.

2. The method according to claim 1, characterized in that, Determining the target noise type from the N noise types based on the probability parameter includes: From the N noise types, the noise type whose probability parameter is greater than the probability threshold value is determined as the target noise type.

3. The method according to any one of claims 1-2, characterized in that, The noise types include noise types classified based on noise sound state, and / or noise types classified based on sound scene.

4. The method according to any one of claims 1-2, characterized in that, The acquisition of the target audio signal to be processed includes: The target audio signal is periodically acquired from the noise frequency to be reduced based on the sampling audio frame interval; The method further includes: Based on the noise reduction results of the target audio signal collected in each cycle of the noise frequency to be reduced, the noise reduction processing of the noise frequency to be reduced is completed.

5. An audio noise reduction device, characterized in that, The device includes: The acquisition module is used to acquire the target audio signal to be processed. The type determination module is used to determine the target sound type corresponding to the valid sound in the target audio signal and the target noise type corresponding to the noise, wherein the target sound type includes speech type and music type; The target noise type is determined as follows: the probability parameters corresponding to the noise and N noise types in the target audio signal are determined respectively. The probability parameters are used to indicate the probability of the corresponding noise type in the target audio signal and the degree of influence on the effective sound. Based on the probability parameters, the target noise type is determined from the N noise types. The number of target noise types is M, 2≤M≤N. The model determination module is used to determine the target noise reduction model corresponding to the target noise type from N noise reduction models. In the N noise reduction models, the N noise reduction models correspond one-to-one with the N noise types, N>1. The N noise reduction models include steady-state noise reduction models, continuous non-steady-state noise reduction models, and transient noise reduction models. The N noise types include steady-state noise, continuous non-steady-state noise, and transient noise. The processing module is configured to perform noise reduction processing on the noise in the target audio signal according to the noise reduction requirements indicated by the target sound type, using the target noise reduction model, to obtain the noise reduction result corresponding to the target audio signal, specifically including: The noise in the target audio signal is denoised using M target noise reduction models corresponding to the target noise types, resulting in M ​​spectral gain data, each of which corresponds one-to-one with one of the M target noise reduction models. The M spectral gain data, the probability parameters corresponding to the M target noise types, and the noise reduction requirements are fused by a fully connected network to obtain the final spectral gain data. The probability parameters are used to identify the degree of influence of the corresponding spectral gain data in the final spectral gain data. The spectral gain data corresponding to the target noise type with a larger probability parameter is amplified, and the spectral gain data corresponding to the target noise type with a smaller probability parameter is reduced. The noise reduction requirement is used to protect the effective sound in the target audio signal. The protection is used to avoid the noise reduction process from affecting the effective sound. The noise reduction requirement includes the noise reduction strength corresponding to the target sound type being speech or music. The noise reduction strength corresponding to speech is greater than the noise reduction strength corresponding to music. If the target sound type is speech, the spectral gain data is squared. If the target sound type is music, the spectral gain data is squared. Based on the final spectral gain data, the noise reduction result corresponding to the target audio signal is determined.

6. The apparatus according to claim 5, characterized in that, The type determination module is specifically used for: From the N noise types, the noise type whose probability parameter is greater than the probability threshold value is determined as the target noise type.

7. The apparatus according to any one of claims 5-6, characterized in that, The noise types include noise types classified based on noise sound state, and / or noise types classified based on sound scene.

8. The apparatus according to any one of claims 5-6, characterized in that, The acquisition module is specifically used for: The target audio signal is periodically acquired from the noise frequency to be reduced based on the sampling audio frame interval; The device also includes an integration module for: Based on the noise reduction results of the target audio signal collected in each cycle of the noise frequency to be reduced, the noise reduction processing of the noise frequency to be reduced is completed.

9. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method described in any one of claims 1-4 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method according to any one of claims 1-4.

11. A computer program product comprising instructions that, when run on a computer, cause the computer to perform the method of any one of claims 1-4.

Images