Audio signal processing methods, devices, equipment, media and program products

Abstract

This application provides an audio signal processing method, apparatus, device, medium, and program product, relating to the technical field of speech noise reduction. The audio signal processing method includes: acquiring an original audio signal; generating a spectrogram based on the original audio signal; decomposing the spectrogram into a basic component matrix and an activation coefficient matrix using nonnegative matrix factorization; separating an initial speech signal using a sparse Bayesian learning model based on the basic component matrix and the activation coefficient matrix, wherein the initial speech signal is represented as a weighted combination of each basic component and activation coefficient; determining an environmental noise feature vector based on the spectrogram data; adjusting the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector; and outputting a target speech signal. This audio signal processing method enables effective suppression of complex environmental noise and accurate extraction of the target speech signal.

Description

Technical Field

[0001] This application relates to the field of speech noise reduction technology, specifically to an audio signal processing method, apparatus, device, medium, and program product. Background Technology

[0002] In the mobile internet era, voice interaction (voice assistants, voice calls) is a core application of mobile devices. However, it is severely interfered with by background noise in noisy scenarios such as public transportation and outdoors, which leads to an increase in voice recognition error rate and a decline in user experience.

[0003] Existing speech denoising solutions have significant limitations in complex scenarios, especially in low signal-to-noise ratio environments. Traditional algorithms struggle to accurately distinguish between target speech and noise, primarily because they rely on fixed parameters or general models, failing to adapt to the dynamic characteristics of noise. Furthermore, different types of noise, such as wind noise and traffic noise, exhibit significant spectral differences, with their intensity and frequency constantly changing. Fixed filters suppress noise in specific frequency bands using preset parameters, making them only suitable for simple environments like offices. In outdoor and subway scenarios, where noise frequencies are wide and highly volatile, they cannot denoise across the entire frequency range, leading to decreased speech clarity. While general denoising models based on deep neural networks (DNNs) or convolutional neural networks (CNNs) offer some adaptability, their parameters are not optimized for specific scenarios. Their performance remains limited in scenarios such as subways (with broadcasts and human voice interference) and strong winds outdoors (where wind noise and ambient sound are superimposed).

[0004] In summary, existing technologies suffer from poor parameter adaptability, weak low signal-to-noise ratio discrimination, and insufficient adaptation to various noise levels. Therefore, there is an urgent need for a noise reduction scheme that can adaptively adjust according to environmental characteristics to improve the stability and accuracy of speech recognition in various complex environments. Summary of the Invention

[0005] This application provides an audio signal processing method, apparatus, device, medium, and program product that can solve the limitations of existing voice noise reduction solutions in dealing with complex environmental scenarios.

[0006] In a first aspect, embodiments of this application provide an audio signal processing method, which includes:

[0007] Acquire the original audio signal and generate a spectrogram based on the original audio signal;

[0008] The spectrogram is decomposed into a basic component matrix and an activation coefficient matrix through nonnegative matrix factorization.

[0009] Based on the basic component matrix and the activation coefficient matrix, an initial speech signal is separated using a sparse Bayesian learning model. The initial speech signal is represented as a weighted combination of each basic component and activation coefficient.

[0010] The environmental noise feature vector is determined based on the spectrogram data. The weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model are adjusted according to the environmental noise feature vector, and the optimized speech signal is output as the target speech signal.

[0011] Optionally, generating a spectrogram based on the original audio signal includes:

[0012] An initial spectrogram is generated based on the original audio signal;

[0013] Calculate the mean and standard deviation of the initial spectrogram at each time frame;

[0014] The spectral data of each time frame is normalized using the mean and standard deviation of each time frame to obtain the normalized spectrum.

[0015] Optionally, the step of decomposing the spectrogram into a basic component matrix and an activation coefficient matrix through nonnegative matrix factorization includes:

[0016] Based on the aforementioned spectrum, the basic component matrix and activation coefficient matrix are initialized as non-negative random matrices;

[0017] With the goal of minimizing the reconstruction error and introducing sparsity constraints, the basic component matrix and the activation coefficient matrix are iteratively optimized using a multiplicative update rule;

[0018] The optimized basic component matrix and activation coefficient matrix are extracted as the target basic component matrix and target activation coefficient matrix.

[0019] Optionally, separating the initial speech signal using a sparse Bayesian learning model based on the basic component matrix and the activation coefficient matrix includes:

[0020] Using the basic component matrix and the activation coefficient matrix as input, a probability distribution model of the target speech signal with the sparsity assumption is established based on the Bayesian inference framework.

[0021] By maximizing the posterior probability estimate, the weighting coefficients of each of the basic components and each of the activation coefficients are determined;

[0022] The initial speech signal is represented as a weighted combination of each basic component and activation coefficient to separate the initial speech signal.

[0023] Optionally, the step of determining the environmental noise feature vector based on spectrogram data, adjusting the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector, and outputting the optimized speech signal as the target speech signal includes:

[0024] The spectrogram is input into a convolutional neural network, which outputs an environmental noise feature vector, which contains noise type and intensity information.

[0025] Based on the environmental noise feature vector, determine the adjusted weight coefficients of each of the basic components in the sparse Bayesian learning model;

[0026] Based on the adjusted weighting coefficients, the audio signals corresponding to each basic component of the initial speech signal are weighted and summed to obtain the optimized speech signal as the target speech signal.

[0027] Secondly, embodiments of this application also provide an audio signal processing apparatus, which includes:

[0028] The generation module is used to acquire the original audio signal and generate a spectrogram based on the original audio signal;

[0029] The decomposition module is used to decompose the spectrogram into a basic component matrix and an activation coefficient matrix through nonnegative matrix decomposition.

[0030] A separation module is used to separate an initial speech signal based on the basic component matrix and the activation coefficient matrix using a sparse Bayesian learning model. The initial speech signal is represented as a weighted combination of each basic component and activation coefficient.

[0031] The adjustment module determines the environmental noise feature vector based on the spectrogram data, adjusts the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector, and outputs the optimized speech signal as the target speech signal.

[0032] Optionally, the generation module is used for:

[0033] An initial spectrogram is generated based on the original audio signal;

[0034] Calculate the mean and standard deviation of the initial spectrogram at each time frame;

[0035] The spectral data of each time frame is normalized using the mean and standard deviation of each time frame to obtain the normalized spectrum.

[0036] Thirdly, embodiments of this application also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the audio signal processing steps as described in the first aspect.

[0037] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the audio signal processing steps described in the first aspect.

[0038] Fifthly, a computer program product is provided, including computer instructions that, when executed by a processor, implement the audio signal processing steps as described in the first aspect.

[0039] The spectrum analysis, nonnegative matrix factorization, and sparse Bayesian learning methods employed in this application lay a core foundation for the accurate extraction of target speech signals. Combined with an environment-aware dynamic correction mechanism, it can effectively suppress background noise and extract target speech in complex environments. At the same time, it optimizes speech recognition and call quality through automatic adaptation of environmental features, ultimately achieving an environment-adaptive audio enhancement effect. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of the audio signal processing method provided in the embodiments of this application;

[0042] Figure 2 This is a flowchart illustrating the separation of a spectrogram into a basic component matrix and an activation coefficient matrix, provided in an embodiment of this application.

[0043] Figure 3 This is a flowchart of adjusting model parameters based on real-time monitored environmental noise, provided in an embodiment of this application.

[0044] Figure 4 This is a structural diagram of the audio signal processing apparatus provided in the embodiments of this application;

[0045] Figure 5 This is a structural diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] This application provides an audio signal processing method.

[0048] See Figure 1 , Figure 1 This is a flowchart of an audio signal processing method provided in an embodiment of this application. The audio signal processing method includes the following steps:

[0049] Step 101: Obtain the original audio signal and generate a spectrogram based on the original audio signal.

[0050] Raw audio signals acquired from complex environments typically have the following characteristics: First, the target speech signal is easily covered by high-intensity background noise, and the two spectra interfere with each other, leading to confusion of signal features; Second, the background noise includes various types of noise such as wind noise, traffic machinery noise, crowd noise, and broadcast noise, forming a composite noise field, and the spectral characteristics of different types of noise are significantly different; Third, the noise has strong dynamic fluctuations, no fixed rules, and a wide and unstable spectral distribution.

[0051] In this step, the process of generating a spectrogram based on the acquired raw audio signal can be implemented using a Short-Time Fourier Transform (STFT). STFT transforms the raw audio signal... The time window is divided into multiple overlapping windows, and a Fourier transform is performed on each window to generate a spectrogram. This method allows time-domain signals to be converted into frequency-domain signals, facilitating subsequent analysis and processing.

[0052] Specifically, the original audio signal The time window is divided into multiple overlapping windows, and a Discrete Fourier Transform (DFT) is performed on each window to obtain the spectrum. It follows the formula below:

[0053] ,

[0054] in, The original audio signal. Indicates time. The position of the current sample point within the window, ranging from 0 to... . The window length is the size of the time window for each Fourier transform. This represents the overlap step size between windows. The Hanning window function is used to reduce spectral leakage. . For frequency variables. imaginary unit , It is a complex exponential function, representing the phase change of different frequency components. This is a spectrum diagram.

[0055] Taking an outdoor running scenario as an example, when runners use voice assistants, background noise includes not only continuous vehicle sounds but also intermittent conversations among other pedestrians. Additionally, the friction noise from running equipment can also be a source of interference. These sound sources overlap in the time domain, and direct noise reduction can easily lead to distortion of the target speech. However, by using STFT to convert the complex time-domain audio signal, which is difficult to distinguish in the time domain, into a spectrogram, each coordinate point in the resulting spectrogram... This represents the signal energy distribution at time t and frequency f, which allows for the clear separation of sound components at different frequencies, thus helping to identify and separate different sound sources.

[0056] To eliminate spectral amplitude differences caused by environmental changes and ensure consistency in subsequent processing, the audio signal processing method in this application embodiment further includes normalizing the generated initial spectrogram after it is generated, so as to make the spectrogram more stable.

[0057] Optionally, generating a spectrogram based on the original audio signal includes:

[0058] An initial spectrogram is generated based on the original audio signal;

[0059] Calculate the mean and standard deviation of the initial spectrogram at each time frame;

[0060] The spectral data of each time frame is normalized using the mean and standard deviation of each time frame to obtain the normalized spectrum.

[0061] Specifically, the mean of each time frame of the generated initial spectrogram is calculated according to the following formulas. and standard deviation Then normalize:

[0062] ,

[0063] ,

[0064] ,

[0065] in, The frequency dimension of the spectrum represents the number of points on the frequency axis. To prevent division by zero of small constants and ensure numerical stability.

[0066] Taking the scenario of running outdoors as an example, the dynamic nature of the environment is particularly prominent: the runner's limb movements may cause the relative distance between the mobile device and noise sources (such as passing vehicles or talking pedestrians) to change in real time, while changes in the environment itself, such as outdoor gusts and fluctuations in traffic density, will also cause drastic fluctuations in the energy intensity of the mixed audio. These factors together result in significant differences in the amplitude of the audio spectrum at different times. This irregular amplitude fluctuation will seriously interfere with the stable recognition of the "target speech-noise" feature by the noise reduction algorithm, and will especially exacerbate the adaptation difficulty of traditional fixed-parameter algorithms.

[0067] In this application, by normalizing the spectrogram, interference caused by dynamic environmental changes can be effectively reduced, providing a stable and consistent spectral data foundation for subsequent noise reduction processing. Specifically, It is a normalized spectrogram that eliminates amplitude fluctuations caused by environmental changes in different time periods. It not only fully preserves the essential differences in frequency distribution between target speech (such as the runner's voice instructions) and background noise (such as wind noise and vehicle noise), but also ensures the consistency of subsequent feature extraction and noise separation processes.

[0068] Step 102: The spectrogram is decomposed into a basic component matrix and an activation coefficient matrix through nonnegative matrix decomposition.

[0069] In this step, the spectrogram generated in step 101 is decomposed into a basic component matrix and an activation coefficient matrix through nonnegative matrix decomposition, that is, the basic component matrix and activation coefficient matrix for the audio signal, thereby extracting the target speech signal more accurately.

[0070] Non-negative matrix factorization (NMF) is a matrix factorization method based on the constraint of data non-negativity. Its decomposition results closely match the energy non-negativity of audio signals. NMF is used to factor high-dimensional spectral data matrices... (The dimension is frequency f × time t) can be decomposed into the product of two low-dimensional matrices (i.e., the fundamental component matrix and the activation coefficient matrix), that is... ≈ × .

[0071] Among them, the basic component matrix Each column corresponds to a "basic component" with a specific spectral profile; these basic components can correspond to a single sound source in a real-world scenario; activation coefficient matrix Each row corresponds to the energy activation intensity of a basic component at different points in time, reflecting the temporal changes in the strength of the sound source.

[0072] This step uses NMF to decompose the spectrogram generated from the original audio signal, extracts the basic spectral features of various sound sources in the audio signal, and clarifies the activation degree of various features at different time points through the activation coefficient matrix, providing accurate feature basis for subsequent targeted preservation of target speech components and suppression of noise components.

[0073] Figure 2 This is a flowchart of decomposing a spectrogram into a basic component matrix and an activation coefficient matrix, provided in an embodiment of this application.

[0074] See Figure 2 Optionally, decomposing the spectrogram into a basic component matrix and an activation coefficient matrix through nonnegative matrix factorization includes the following steps:

[0075] Step 201: Based on the spectrum, initialize the basic component matrix and activation coefficient matrix as non-negative random matrices;

[0076] Step 202: With the goal of minimizing the reconstruction error and introducing sparsity constraints, the basic component matrix and the activation coefficient matrix are iteratively optimized using the multiplication update rule;

[0077] Step 203: Extract the optimized basic component matrix and activation coefficient matrix as the target basic component matrix and target activation coefficient matrix.

[0078] Specifically, first, the basic component matrix and activation coefficient matrix are initialized.

[0079] definition The basic component matrix, where It is the frequency dimension of the spectrum. It refers to the quantity of the basic components. (Definition) Let be the activation coefficient matrix, where It is the time dimension of the spectrum.

[0080] initialization and A non-negative random matrix:

[0081] , ,

[0082] in, Index of basic components, ranging from 1 to . The elements in the basic component matrix represent the first... The first frequency dimension One basic component. Let be an element in the activation coefficient matrix, representing the th . The basic components in the first The level of activation on each time frame.

[0083] Next, an iterative optimization process is performed on the basic component matrix and the activation coefficient matrix.

[0084] To minimize the reconstruction error and introduce sparsity constraints, a multiplicative update rule is used to iteratively optimize the fundamental component matrix. and activation coefficient matrix The details are as follows:

[0085] The goal is to minimize the reconstruction error, and a sparsity constraint is introduced:

[0086] ,

[0087] in, To reconstruct the spectrogram. It is the Frobenius norm, which measures the distance between matrices. It uses the L1 norm to promote sparsity. Hyperparameters for controlling sparsity.

[0088] Use the multiplication update rule:

[0089] ,

[0090] in, These are the elements in the updated basic component matrix. It is the product of the transpose of the activation coefficient matrix and the normalized spectrum. The product result is used to reconstruct the spectrogram. It is the identity matrix. This is a regularization term to prevent overfitting.

[0091] ,

[0092] in, These are the elements in the updated basic component (activation coefficient) matrix. It is the product of the transpose of the basic component matrix and the normalized spectrum. The product result is used to reconstruct the spectrogram. This is a regularization term to prevent overfitting.

[0093] Continuing with the example of running outdoors, as mentioned earlier, background noise includes vehicle sounds and conversations from other pedestrians. Using NMF (Natural Component Modeling), these sound sources are separated into basic component matrices, allowing for more accurate extraction of the target speech signal. It can represent different sound sources (such as human voices, vehicle sounds), and This indicates the activation level of these sounds at different points in time. Specifically, Each column vector in the vector represents a basic component, such as the low-frequency basic contour of vehicle traffic sounds, the mid-to-high frequency contour of pedestrian conversations, and the feature contour of the target speech, respectively. Each row vector in the vector represents how the basic component changes over time.

[0094] After the above iterative optimization process, the optimized basic component matrix and the activation coefficient matrix are extracted as the target basic component matrix and the target activation coefficient matrix.

[0095] Step 103: Based on the basic component matrix and the activation coefficient matrix, a sparse Bayesian learning model is used to separate the initial speech signal, which is represented as a weighted combination of each basic component and activation coefficient.

[0096] In this step, based on the basic component matrix and activation coefficient matrix extracted in step 102, the initial speech signal is separated using a Sparse Bayesian Learning (SBL) model. SBL models the probability distribution of each basic component using a Bayesian inference framework and introduces the sparsity assumption to extract the target speech signal more accurately.

[0097] Optionally, based on the basic component matrix and the activation coefficient matrix, separating the initial speech signal using a sparse Bayesian learning model includes:

[0098] Using the basic component matrix and the activation coefficient matrix as input, a probability distribution model of the target speech signal with the sparsity assumption is established based on the Bayesian inference framework.

[0099] By maximizing the posterior probability estimate, the weighting coefficients of each of the basic components and each of the activation coefficients are determined;

[0100] The initial speech signal is represented as a weighted combination of each basic component and activation coefficient to separate the initial speech signal.

[0101] Specifically, a Bayesian inference framework is used to model the probability distribution of each basic component, and a sparsity assumption is introduced:

[0102] ,

[0103] in, The target speech signal. Let be the probability distribution of the target speech. It uses the L1 norm to promote sparsity. Let V be the variance of the target speech. Hyperparameters for controlling sparsity. The squared error between the target speech signal and the reconstructed spectrogram.

[0104] By maximizing the posterior probability estimate and suppressing other components, the weighting coefficients of each of the basic components and each of the activation coefficients are determined:

[0105] ,

[0106] in, These are the weighting coefficients, assigned based on the posterior probability. is a column vector in the activation coefficient matrix. These are column vectors in the basic component matrix.

[0107] Assuming you're running outdoors, background noise includes vehicle sounds and conversations from other pedestrians. Using the SBL model, the system can prioritize processing frequency bands less affected by wind noise, thus improving speech recognition accuracy. It can dynamically adjust based on the intensity of the wind noise, giving higher weight to frequencies with lower wind noise levels. Specifically, This indicates the importance of each basic component; a higher weight means that the component is more likely to be part of the target speech signal.

[0108] Step 104: Determine the environmental noise feature vector based on the spectrogram data, adjust the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector, and output the optimized speech signal as the target speech signal.

[0109] In this step, a clearer voice interaction experience is provided by monitoring environmental noise patterns in real time and adjusting model parameters based on noise characteristics.

[0110] Figure 3 This is a flowchart of adjusting model parameters based on real-time monitored environmental noise, provided in an embodiment of this application.

[0111] like Figure 3 As shown, optionally, the step of determining the environmental noise feature vector based on spectrogram data, adjusting the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector, and outputting the optimized speech signal as the target speech signal includes:

[0112] Step 301: Input the spectrum into a convolutional neural network and output an environmental noise feature vector, wherein the environmental noise feature vector contains noise type and intensity information;

[0113] Step 302: Determine the adjusted weight coefficients of each of the basic components in the sparse Bayesian learning model based on the environmental noise feature vector.

[0114] Step 303: Based on the adjusted weighting coefficients, the audio signals corresponding to each basic component of the initial speech signal are weighted and summed to obtain the optimized speech signal as the target speech signal.

[0115] Specifically, a convolutional neural network (CNN) is used to classify environmental noise:

[0116] ,

[0117] in, This is the output environmental noise feature vector, which contains information such as noise type and intensity. Input spectrogram data.

[0118] Here, the CNN model is able to better capture local features in the spectrogram, improving the accuracy of noise classification.

[0119] Next, based on the environmental noise feature vector Adjust the weight allocation in the SBL model to adapt to different noise environments.

[0120] ,

[0121] in, For the first The weights of each basic component. For the first The noise feature vectors corresponding to the basic components.

[0122] Finally, the optimized speech signal is output as the target speech signal, which reflects the weighted combination of different basic components to meet the speech enhancement needs in the current environment.

[0123] ,

[0124] in, This is the optimized speech signal. For the first The weights of each basic component. For the first The signals corresponding to each basic component.

[0125] Continuing with the example of running outdoors, as mentioned earlier, background noise includes vehicle sounds and conversations from other pedestrians. When strong vehicle noise is detected, the system automatically adjusts the weighting, prioritizing frequency bands less affected by vehicle noise. Specifically, This represents the weight of each basic component; a higher weight means that the component is more likely to be the target speech signal in the current context. The final output is an optimized speech signal, reflecting a weighted combination of different basic components to adapt to the speech enhancement needs of the current environment. Through the environment-aware dynamic correction layer in this step, the system can adjust parameters according to real-time noise characteristics, providing a clear voice interaction experience.

[0126] In the audio signal processing method of this application embodiment, the core foundation for accurate extraction of target speech signals is laid by using spectrum analysis, non-negative matrix factorization and sparse Bayesian learning methods; combined with the dynamic correction mechanism based on environment awareness, background noise can be effectively suppressed and target speech can be extracted in complex environments, while speech recognition and call quality are optimized by automatically adapting environmental features, and finally, an environment-adaptive audio enhancement effect is achieved.

[0127] Figure 4 This is a structural diagram of an audio signal processing apparatus provided in an embodiment of this application. Figure 4 As shown, the device 400 includes:

[0128] Generation module 401 is used to acquire the original audio signal and generate a spectrum diagram based on the original audio signal;

[0129] The decomposition module 402 is used to decompose the spectrum into a basic component matrix and an activation coefficient matrix through non-negative matrix decomposition.

[0130] The separation module 403 is used to separate the initial speech signal based on the basic component matrix and the activation coefficient matrix using a sparse Bayesian learning model. The initial speech signal is represented as a weighted combination of each basic component and activation coefficient.

[0131] The adjustment module 404 is used to determine the environmental noise feature vector based on the spectrogram data, adjust the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector, and output the optimized speech signal as the target speech signal.

[0132] Optionally, the generation module 401 is also used for:

[0133] An initial spectrogram is generated based on the original audio signal;

[0134] Calculate the mean and standard deviation of the initial spectrogram at each time frame;

[0135] The spectral data of each time frame is normalized using the mean and standard deviation of each time frame to obtain the normalized spectrum.

[0136] Optionally, the decomposition module 402 is also used for:

[0137] Based on the aforementioned spectrum, the basic component matrix and activation coefficient matrix are initialized as non-negative random matrices;

[0138] With the goal of minimizing the reconstruction error and introducing sparsity constraints, the basic component matrix and the activation coefficient matrix are iteratively optimized using a multiplicative update rule;

[0139] The optimized basic component matrix and activation coefficient matrix are extracted as the target basic component matrix and target activation coefficient matrix.

[0140] Optionally, the separation module 403 is also used for:

[0141] Using the basic component matrix and the activation coefficient matrix as input, a probability distribution model of the target speech signal with the sparsity assumption is established based on the Bayesian inference framework.

[0142] By maximizing the posterior probability estimate, the weighting coefficients of each of the basic components and each of the activation coefficients are determined;

[0143] The initial speech signal is represented as a weighted combination of each basic component and activation coefficient to separate the initial speech signal.

[0144] Optionally, the adjustment module 404 is also used for:

[0145] The spectrogram is input into a convolutional neural network, which outputs an environmental noise feature vector, which contains noise type and intensity information.

[0146] Based on the environmental noise feature vector, determine the adjusted weight coefficients of each of the basic components in the sparse Bayesian learning model;

[0147] Based on the adjusted weighting coefficients, the audio signals corresponding to each basic component of the initial speech signal are weighted and summed to obtain the optimized speech signal as the target speech signal.

[0148] The audio signal processing device 400 provided in this application embodiment is capable of implementing... Figure 1 The entire technical process of the method shown, and the same technical effect, will not be described again here to avoid repetition.

[0149] The audio signal processing device 400 in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. Non-mobile electronic devices can also be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not specifically limit the specific type of electronic device.

[0150] Optionally, such as Figure 5 As shown, this application embodiment also provides an electronic device 500, including a processor 501 and a memory 502. The memory 502 stores a program or instructions that can run on the processor 501. When the program or instructions are executed by the processor 501, they implement the various steps of the above-described audio signal processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0151] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0152] Furthermore, from the perspective of needing voice interaction functions, the electronic devices in the embodiments of this application include not only traditional mobile devices such as mobile phones and tablets, and IoT devices such as smart wearable devices and smart home devices, but also transportation navigation devices and voice control devices in industrial environments.

[0153] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described audio signal processing method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0154] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes of the audio signal processing method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0155] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0157] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An audio signal processing method, characterized in that, include: Acquire the original audio signal and generate a spectrogram based on the original audio signal; The spectrogram is decomposed into a basic component matrix and an activation coefficient matrix through nonnegative matrix factorization. Based on the basic component matrix and the activation coefficient matrix, an initial speech signal is separated using a sparse Bayesian learning model. The initial speech signal is represented as a weighted combination of each basic component and activation coefficient. The environmental noise feature vector is determined based on the spectrogram data. The weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model are adjusted according to the environmental noise feature vector, and the optimized speech signal is output as the target speech signal.

2. The method according to claim 1, characterized in that, The generation of the spectrogram based on the original audio signal includes: An initial spectrogram is generated based on the original audio signal; Calculate the mean and standard deviation of the initial spectrogram at each time frame; The spectral data of each time frame is normalized using the mean and standard deviation of each time frame to obtain the normalized spectrum.

3. The method according to claim 1, characterized in that, The step of decomposing the spectrogram into a basic component matrix and an activation coefficient matrix through nonnegative matrix factorization includes: Based on the aforementioned spectrum, the basic component matrix and activation coefficient matrix are initialized as non-negative random matrices; With the goal of minimizing the reconstruction error and introducing sparsity constraints, the basic component matrix and the activation coefficient matrix are iteratively optimized using a multiplicative update rule; The optimized basic component matrix and the activation coefficient matrix are extracted as the target basic component matrix and the target activation coefficient matrix.

4. The method according to claim 1, characterized in that, The separation of the initial speech signal using a sparse Bayesian learning model based on the basic component matrix and the activation coefficient matrix includes: Using the basic component matrix and the activation coefficient matrix as input, a probability distribution model of the target speech signal with the sparsity assumption is established based on the Bayesian inference framework. The weighting coefficients of each of the basic components and each of the activation coefficients are determined by maximizing the posterior probability estimate; The initial speech signal is represented as a weighted combination of each basic component and activation coefficient to separate the initial speech signal.

5. The method according to claim 4, characterized in that, The process of determining the environmental noise feature vector based on spectrogram data, adjusting the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector, and outputting the optimized speech signal as the target speech signal includes: The spectrogram is input into a convolutional neural network, which outputs an environmental noise feature vector, which contains noise type and intensity information. Based on the environmental noise feature vector, determine the adjusted weight coefficients of each of the basic components in the sparse Bayesian learning model; Based on the adjusted weighting coefficients, the audio signals corresponding to each basic component of the initial speech signal are weighted and summed to obtain the optimized speech signal as the target speech signal.

6. An audio signal processing device, characterized in that, include: The generation module is used to acquire the original audio signal and generate a spectrogram based on the original audio signal; The decomposition module is used to decompose the spectrogram into a basic component matrix and an activation coefficient matrix through nonnegative matrix decomposition. A separation module is used to separate an initial speech signal based on the basic component matrix and the activation coefficient matrix using a sparse Bayesian learning model. The initial speech signal is represented as a weighted combination of each basic component and activation coefficient. The adjustment module is used to determine the environmental noise feature vector based on the spectrogram data, adjust the weight coefficients of each basic component and activation coefficient in the sparse Bayesian learning model according to the environmental noise feature vector, and output the optimized speech signal as the target speech signal.

7. The audio signal processing apparatus according to claim 6, characterized in that, The generation module is used for: An initial spectrogram is generated based on the original audio signal; Calculate the mean and standard deviation of the initial spectrogram at each time frame; The spectral data of each time frame is normalized using the mean and standard deviation of each time frame to obtain the normalized spectrum.

8. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the audio signal processing method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the audio signal processing method as described in any one of claims 1 to 5.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the audio signal processing method as described in any one of claims 1 to 5.

Images