Audio processing method and device, electronic device, and storage medium

Abstract

Embodiments of the present disclosure provide an audio processing method and device, an electronic device and a storage medium. The method comprises: obtaining an original spectrum of each audio frame in a sequence of audio frames; determining a history frame fusion weight of a frequency point in each original spectrum according to a spectrum value of the frequency point in a plurality of continuous audio frames; wherein the plurality of audio frames comprise a current frame and at least one history frame corresponding to the current frame; the history frame fusion weight is related to a spectrum value stability degree of the frequency point in the plurality of audio frames; fusing the original spectrum value of the frequency point in the current frame with a history spectrum value in the history frame based on the history frame fusion weight to obtain a fusion spectrum value of the frequency point in the current frame; generating a noise reduction spectrum of each audio frame based on the fusion spectrum value of all frequency points in each original spectrum; and generating a target audio signal based on the noise reduction spectrum of each audio frame. The method can realize adaptive noise reduction of the audio signal and improve noise reduction precision.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of audio signal processing technology, specifically to audio processing methods and apparatus, electronic devices and storage media, and computer program products. Background Technology

[0002] In the field of audio signal processing technology, noise suppression is an important branch of technology, aiming to suppress noise in audio signals containing speech. Currently, audio noise suppression techniques mainly focus on spectral subtraction based on frequency domain analysis, statistical model filtering, and speech enhancement based on deep learning. Summary of the Invention

[0003] Embodiments of this disclosure provide audio processing methods and apparatus, electronic devices, computer-readable storage media, and computer program products.

[0004] In a first aspect, embodiments of this disclosure provide an audio processing method, comprising: acquiring the original spectrum of each audio frame in a frame sequence of an audio signal; for any frequency point in each original spectrum, determining a historical frame fusion weight of the frequency point based on the original spectral values ​​of the frequency point in multiple consecutive audio frames; wherein the multiple audio frames include a current frame and at least one historical frame corresponding to the current frame; the historical frame fusion weight is related to the stability of the spectral values ​​of the frequency point in multiple audio frames; based on the historical frame fusion weight, fusing the original spectral values ​​of the frequency point in the current frame with the historical spectral values ​​in the historical frames to obtain the fused spectral values ​​of the frequency point in the current frame; generating a denoised spectrum for each audio frame based on the fused spectral values ​​of all frequency points in each original spectrum; and generating a target audio signal based on the denoised spectrum for each audio frame.

[0005] In some embodiments, determining the historical frame fusion weight of a frequency point based on its original spectral values ​​in multiple consecutive audio frames includes: for each audio frame in the multiple audio frames, determining the spectral value change of the frequency point in the audio frame based on the original spectral value of the frequency point in the audio frame and the original spectral value in the previous audio frame; determining a target number of consecutive stable frames based on the spectral value change of the frequency point in each audio frame in the multiple audio frames and a preset stability threshold; and determining the historical frame fusion weight of the frequency point based on the target number of consecutive stable frames.

[0006] In some embodiments, determining the target number of consecutive stable frames based on the spectral value change of the frequency point in each of the multiple audio frames and a preset stability threshold includes: for each of the multiple audio frames, if the spectral value change of the audio frame is less than the stability threshold, incrementing the number of recorded consecutive stable frames by 1; if the spectral value change of the audio frame is greater than or equal to the stability threshold, resetting the number of recorded consecutive stable frames; and if the audio frame is the current frame, determining the number of recorded consecutive stable frames corresponding to the current frame as the target number of consecutive stable frames.

[0007] In some embodiments, determining the historical frame fusion weights for a frequency point based on a target number of consecutive stable frames includes: determining the historical frame fusion weights based on a target number of consecutive stable frames and a threshold number of consecutive stable frames.

[0008] In some embodiments, determining the historical frame fusion weight of a frequency point based on a target continuous stable frame number includes: setting the value of the historical frame fusion weight in a first weight range in response to the target continuous stable frame number being greater than or equal to a preset stable frame number threshold; and setting the value of the historical frame fusion weight in a second weight range in response to the target continuous stable frame number being less than the preset stable frame number threshold; wherein the upper limit of the weight in the second weight range is less than the lower limit of the weight in the first weight range.

[0009] In some embodiments, based on the historical frame fusion weight, the original spectral value of the frequency point in the current frame is fused with the historical spectral value in the historical frame to obtain the fused spectral value of the frequency point in the current frame, including: determining the fusion weight of the current frame according to the difference between the preset fusion weight and the historical frame fusion weight; and weightedly fusing the original spectral value of the frequency point in the current frame with the fused spectral value in the previous audio frame of the current frame based on the fusion weight of the current frame and the historical frame fusion weight to obtain the fused spectral value of the frequency point in the current frame.

[0010] In some embodiments, obtaining the original spectrum of each audio frame in the frame sequence of an audio signal includes: performing short-time framing and windowing processing on the audio signal to obtain a frame sequence; performing Fourier transform on each audio frame in the frame sequence to obtain a frequency domain representation of each audio frame; and obtaining the original spectrum of each audio frame based on the frequency domain representation.

[0011] In some embodiments, the total number of frequency points included in each raw spectrum is determined based on a preset sampling rate and Fourier points.

[0012] In some embodiments, generating a target audio signal based on the denoised spectrum of each audio frame includes: performing an inverse Fourier transform on the denoised spectrum of each audio frame to convert it into a time-domain denoised signal; and performing windowing and overlapping addition processing on the time-domain denoised signal to generate the target audio signal.

[0013] Secondly, embodiments of this disclosure provide an audio processing apparatus, comprising: a signal processing module configured to acquire the original spectrum of each audio frame in a frame sequence of an audio signal; a weight determination module configured to determine a historical frame fusion weight for any frequency point in each original spectrum based on the original spectral values ​​of the frequency point in multiple consecutive audio frames; wherein the multiple audio frames include a current frame and at least one historical frame corresponding to the current frame; the historical frame fusion weight is related to the stability of the spectral values ​​of the frequency point in the multiple audio frames; a spectral value fusion module configured to fuse the original spectral values ​​of the frequency point in the current frame with the historical spectral values ​​in the historical frames based on the historical frame fusion weight to obtain the fused spectral values ​​of the frequency point in the current frame; a spectrum generation module configured to generate a denoised spectrum for each audio frame based on the fused spectral values ​​of all frequency points in each original spectrum; and a signal generation module configured to generate a target audio signal based on the denoised spectrum of each audio frame.

[0014] Thirdly, embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the audio processing method as mentioned in any of the first aspects.

[0015] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the audio processing method as mentioned in any of the first aspects.

[0016] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements an audio processing method as mentioned in any of the first aspects.

[0017] The audio processing method and apparatus provided in the embodiments of this disclosure propose a dynamic frequency domain fusion noise suppression mechanism based on the time stability analysis of frequency components. By evaluating the amplitude of spectral value changes of each frequency point in multiple consecutive audio frames, and dynamically adjusting the fusion weight of the current frame and the historical frames according to the determined historical frame fusion weight, for frequency components with time stability, i.e., frequency points, efficient suppression of stable noise frequency points is achieved by increasing the historical frame fusion weight. This enables adaptive denoising of the speech spectrum without relying on complex models or large amounts of computing resources, thereby improving the noise reduction accuracy.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an exemplary system architecture diagram to which one embodiment of this disclosure can be applied; Figure 2 This is a flowchart of an embodiment of the audio processing method according to the present disclosure; Figure 3 This is an exemplary flowchart of the audio processing method according to this disclosure in an application scenario; Figure 4 This is a schematic diagram illustrating the composition of an audio processing apparatus according to an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0020] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] While existing audio noise suppression methods have solved some noise problems in specific scenarios, they have some technical shortcomings, resulting in poor denoising effects or limitations in practical deployment that restrict their application scenarios. For example, noise suppression techniques based on spectral subtraction often rely on static thresholds or fixed noise reduction coefficients, making it difficult to finely process the temporal variation characteristics of different frequency points. They also have poor adaptability to non-stationary noise and slowly changing background noise, and are prone to over- or under-subtraction due to noise estimation errors. Wiener filtering uses statistical models to construct frequency domain gains, applying a uniform statistical model to all frequency points, lacking differentiated time dimension control. Under non-stationary noise, it is prone to lag phenomena such as untimely noise suppression or excessive speech smoothing. Deep learning-based speech enhancement methods rely on a large amount of labeled data for training. When data coverage is insufficient, their generalization ability is poor, and they have high computational complexity and large storage requirements, making them difficult to deploy and run stably in low-power devices, real-time communication systems, or embedded environments.

[0023] Currently, accurate noise reduction of raw audio signals is in high demand in various low-power devices such as wearable devices. Simply using software-level noise reduction algorithms has limited effectiveness; therefore, in practical applications, dedicated noise reduction chips are preferred for more efficient hardware-level processing. For these low-power devices, improving noise reduction performance without requiring high power consumption is an important research direction in the application of low-power chips.

[0024] The audio processing method and apparatus provided according to the embodiments of this disclosure at least partially solve the above-mentioned problems existing in the related art.

[0025] The present disclosure will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present disclosure can be combined with each other. Furthermore, unless explicitly limited or contradicted by the context, the specific steps included in the methods described in this disclosure are not limited to the order in which they are described, but can be performed in any order or in parallel.

[0026] Figure 1 An exemplary system architecture 100 is shown that can be applied to embodiments of the audio processing methods or audio processing apparatus disclosed herein.

[0027] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0028] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays that support receiving messages from the server, including but not limited to smartphones, tablets, e-book readers, electronic players, laptops, and desktop computers, etc.

[0029] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., multiple software modules used to provide distributed services), or as a single software program or software module, such as a client / server application or a web browser application. No specific limitations are made here.

[0030] Server 105 can be a server providing various services. Taking an audio processing application that can provide audio processing services as an example, when running an audio processing application, server 105 can achieve the following effects: First, obtain the original spectrum of each audio frame in the frame sequence of the audio signal; then, for any frequency point in each original spectrum, determine the historical frame fusion weight of the frequency point based on the original spectral values ​​of the frequency point in multiple consecutive audio frames; wherein, the multiple audio frames include the current frame and at least one historical frame corresponding to the current frame; the historical frame fusion weight is related to the stability of the spectral values ​​of the frequency point in multiple audio frames; next, based on the historical frame fusion weight, fuse the original spectral value of the frequency point in the current frame with the historical spectral value in the historical frame to obtain the fused spectral value of the frequency point in the current frame; then, based on the fused spectral values ​​of all frequency points in each original spectrum, generate the denoised spectrum of each audio frame; finally, generate the target audio signal based on the denoised spectrum of each audio frame.

[0031] It's important to note that a server can be either hardware or software. When a server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When a server is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here. A server can also be a server for a distributed system, or a server integrated with blockchain technology. A server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0032] It should be noted that the audio processing method provided in this embodiment can be executed by server 105 or by terminal devices 101, 102, and 103. Accordingly, the audio processing device can be located in server 105 or in terminal devices 101, 102, and 103.

[0033] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0034] Figure 2 A flow 200 is shown as an embodiment of the audio processing method according to this disclosure. For example... Figure 2 As shown, process 200 includes the following steps: Step 201: Obtain the original spectrum of each audio frame in the frame sequence of the audio signal.

[0035] This step is intended for the execution subject of the audio processing method, such as... Figure 1The terminal devices 101, 102, 103 or server 105 shown in the diagram divide the input audio signal to be processed into frames to generate a frame sequence including multiple audio frames, and perform time-domain to frequency-domain conversion on each audio frame in the frame sequence to obtain the original spectrum of each audio frame.

[0036] According to the audio processing method of this disclosure, the audio signal to be processed can be a noisy speech signal, such as a speech signal that is continuous in time acquired in speech task scenarios such as speech recognition, voiceprint recognition, and conference recording.

[0037] In some implementations, the raw spectrum of each audio frame may include multiple frequency points, each corresponding to a specific frequency component. The raw spectrum of each audio frame may be, for example, an amplitude spectrum or a power spectrum.

[0038] Step 202: For any frequency point in each original spectrum, determine the historical frame fusion weight of the frequency point based on the original spectral values ​​of the frequency point in multiple consecutive audio frames.

[0039] According to embodiments of this disclosure, for any frequency point in each original spectrum, the continuous stability of that frequency point can be determined based on the degree of spectral value change of that frequency point across multiple consecutive audio frames, and then its historical frame fusion weight can be determined based on the continuous stability of the frequency point. The multiple consecutive audio frames may include the current frame and at least one corresponding historical frame. The historical frame fusion weight is related to the stability of the spectral value of that frequency point across multiple audio frames. This historical frame fusion weight is used to enhance and suppress noise frequency components.

[0040] In some alternative implementations, for each frequency point, if the original spectral value of the frequency point fluctuates significantly over a preset time period, such as in multiple consecutive audio frames, it indicates that the frequency point is currently less stable; conversely, if its original spectral value fluctuates less over time, it indicates that the frequency point is currently relatively stable.

[0041] Furthermore, the historical frame fusion weight can be set based on the continuous stability of each frequency point. For example, the magnitude of the historical frame fusion weight can be inversely proportional to the stability of the spectral value change of that frequency point across multiple consecutive audio frames. For frequency points with high continuous stability, indicating a higher probability of noise, a higher historical frame fusion weight can be set, making the output closer to historical statistical characteristics and thus significantly suppressing random noise fluctuations. For frequency points with low continuous stability, a lower historical frame fusion weight can be set, preserving more of the speech details of the current frame. Through this frequency-level dynamic frequency domain fusion, both noise suppression and speech fidelity can be achieved.

[0042] Step 203: Based on the historical frame fusion weight, the original spectral value of the frequency point in the current frame is fused with the historical spectral value in the historical frames to obtain the fused spectral value of the frequency point in the current frame.

[0043] In some optional implementations, for each frequency point, the original spectral value of the current frame and the historical spectral values ​​of historical frames can be weighted and fused according to the historical frame fusion weights to obtain the fused spectral value of that frequency point in the current frame. The historical spectral values ​​of historical frames can be either the original spectral values ​​or the fused spectral values ​​of historical frames.

[0044] In some alternative implementations, for each frequency point, the original spectral value of the current frame can be weighted and fused with the original spectral value of the previous frame according to the historical frame fusion weight. Alternatively, the original spectral value of the current frame can be weighted and fused with the average of the original spectral values ​​of multiple historical frames corresponding to the current frame.

[0045] In some optional implementations, for each frequency point, the original spectral value of the current frame can be weighted and fused with the fused spectral value of the previous frame according to the historical frame fusion weights to obtain the fused spectral value of that frequency point in the current frame. For example, the current frame is the [number missing]th [frame missing]. n Frame, the previous frame is the first n -1 frame. This can be understood as the fused spectral value of the previous frame, i.e., the frame number... n The fused spectral value of frame -1 is based on the frequency point at the [number]th frame. n -1 frame's historical frame fusion weight, for this frame n The original spectrum of frame -1 and the previous frame, i.e., the first frame... n The fusion spectral values ​​of -2 frames are obtained by weighted fusion. n For frame number, n The value can be a positive integer.

[0046] Step 204: Generate the noise-reduced spectrum for each audio frame based on the fused spectral values ​​of all frequency points in each original spectrum.

[0047] For each audio frame, the denoised spectrum can be obtained from the fused spectral values ​​of all frequency points in the original spectrum of the audio frame. This denoised spectrum can be understood as the spectrum of the original spectrum after denoising.

[0048] Step 205: Generate the target audio signal based on the noise reduction spectrum of each audio frame.

[0049] In some implementations, the final target audio signal can be generated by converting the noise reduction spectrum of each audio frame to the frequency domain.

[0050] Based on the audio processing method of the above embodiments of this disclosure, a dynamic frequency domain fusion noise suppression mechanism based on the temporal stability analysis of frequency components is proposed. This mechanism evaluates the degree of spectral change of each frequency point across multiple consecutive audio frames and dynamically adjusts the fusion weights of the current frame and historical frames according to the degree of spectral change of each frequency point. For noise frequencies with significant temporal stability, efficient suppression of stable noise frequencies is achieved by increasing the fusion weights of historical frames, thereby improving noise reduction accuracy. This embodiment performs dynamic frequency domain fusion based on the statistical stability of frequency points over time. It does not require building a noise model or training a neural network, nor does it require noise type labels, training data, or inference devices. It has low computational complexity and can achieve adaptive denoising of the speech spectrum without relying on complex models or large amounts of computational resources, and it also has strong scene generalization capabilities.

[0051] To facilitate understanding and to highlight the effects of the audio processing method provided in this disclosure in practical application scenarios as much as possible, the audio processing method provided in this disclosure will be described by example below.

[0052] According to some exemplary embodiments, based on the above embodiments, the execution subject of the audio processing method, such as a terminal device or a server, can obtain the original spectrum of each audio frame in the frame sequence of the audio signal in the following manner: First, perform short-time framing and windowing processing on the input audio signal to obtain a frame sequence; then, perform Fourier transform on each audio frame in the frame sequence to obtain the frequency domain representation of each audio frame; then, obtain the original spectrum of each audio frame based on the frequency domain representation.

[0053] In practical applications, audio signals are usually time-varying. For changing, non-stationary signals, to facilitate subsequent analysis and processing, the time-domain audio signal is first divided into short-time frames. The audio is divided into a series of short, relatively stable segments, each segment being called an audio frame. , forming a frame sequence { }.in, n For frame number, n The value can be a positive integer.

[0054] To reduce spectral leakage caused by discontinuities at frame boundaries, windowing can be applied to each audio frame in the frame sequence to reduce abrupt changes and discontinuities between frames. For example, each audio frame is multiplied by a window function, such as a Hamming window or Hanning window, to obtain a windowed audio frame. Both the unwindowed and windowed audio frames are time-domain frames. The specific method for windowing the audio frames is shown below: ; in, For the first n Frame audio frame, w ( t ) is a window function. The audio frame after windowing.

[0055] In addition, to ensure temporal smoothness, each audio frame can overlap with its preceding and following frames by a certain proportion. The formula for calculating the overlap proportion is as follows: ; in, overlap _ ratio The overlap ratio; R Frame shift represents the time interval between the start or end of two adjacent frames; L The frame length is the duration of each frame.

[0056] Then, for each windowed audio frame in the frame sequence This involves converting the audio from the time domain to the frequency domain. For example, a Fast Fourier Transform (FFT) can be performed on each frame to obtain the frequency domain representation of each audio frame. The details are as follows: ; in, For each audio frame, the frequency domain representation is defined by FFT, which stands for Fast Fourier Transform.

[0057] Frequency domain representation is a descriptive form of an audio signal mapped from the time domain to the frequency domain, providing a foundation for subsequent frequency point analysis and frequency domain fusion. A complete frequency domain representation of each audio frame typically includes information such as amplitude and phase.

[0058] Furthermore, the original spectrum of each audio frame, and the original spectral value of each frequency point in the original spectrum, can be obtained from the frequency domain representation. The original spectrum of each audio frame may include multiple frequency points, each corresponding to a specific frequency component, for example... Representing the k One frequency component, k The value can be a positive integer. The total number of frequency points included in each original spectrum can be determined based on a preset sampling rate and the number of Fourier points. The number of Fourier points refers to the number of points in the Fourier transform, and the sampling rate determines the frequency range of the spectrum.

[0059] According to some embodiments of this disclosure, the original spectrum of each audio frame can be an amplitude spectrum or a power spectrum; correspondingly, the original spectral value of each frequency point can be an amplitude spectrum value or a power spectrum value. For example, the amplitude spectrum of the audio frame can be obtained by taking the modulus of the frequency domain representation of each audio frame. The amplitude spectrum may include the amplitude spectrum value at each frequency point. Alternatively, the power spectrum of the audio frame can be obtained by squared the amplitude spectrum, and then the power spectrum value of each frequency point can be obtained.

[0060] Based on the above embodiments, and according to some exemplary implementations of this disclosure, for Figure 2 The above step 202 can be specifically implemented as steps 2021-2023.

[0061] Step 2021: For each audio frame in the multiple audio frames, determine the change value of the spectral value of any frequency point in the audio frame based on the original spectral value of any frequency point in the audio frame and the original spectral value of any frequency point in the previous audio frame.

[0062] For each frequency point in the original spectrum of each audio frame Based on its original spectral values ​​across multiple consecutive audio frames, the spectral value change of that frequency point in each audio frame is determined. These multiple consecutive audio frames may include the current frame and the preceding frames corresponding to that current frame. m A historical frame, m The value can be a positive integer greater than 1. That is, the change in spectral value of the frequency point in the current frame relative to the previous frame is calculated, and the change in spectral value of the previous frame is calculated. m The spectral change value of each historical frame relative to its previous frame. The spectral change value can be, for example, the difference between the original spectral values ​​of the frequency point in two adjacent audio frames, used to measure the stability or degree of spectral change of the frequency point in two adjacent audio frames.

[0063] For example, the change in spectral value of the current frame relative to the previous frame. As shown below: ; in, The original spectral values ​​for the current frame. This is the original spectral value of the previous frame of the current frame.

[0064] That is, the change in spectral value of the current frame relative to the previous frame. It is based on the raw spectral values ​​of the current frame. Compared with the original spectral value of the previous frame The difference between them is determined.

[0065] It should be noted that the original spectral values ​​can be either amplitude spectral values ​​corresponding to each frequency point or power spectral values; this disclosure does not impose any specific restrictions on this. The amplitude spectral values ​​reflect the amplitude intensity of each frequency component, while the power spectral values ​​reflect the energy distribution of each frequency component. Essentially, the power spectral values ​​and amplitude spectral values ​​have a square relationship.

[0066] Step 2022: Determine the target number of consecutive stable frames based on the spectral value change of the frequency point in each audio frame and the preset stability threshold.

[0067] By defining a stability threshold θ The stability of a frequency point is determined based on the spectral variation in each audio frame, and the number of consecutive stable frames can be counted based on the stability determination results for that frequency point in each audio frame. This number of consecutive stable frames is used as the cross-frame stability index of the frequency point, representing the number of times the frequency point remains stable across multiple consecutive audio frames.

[0068] According to some exemplary implementations, for any frequency point in each audio frame, the spectral value change value of that point in each of several consecutive audio frames can be statistically analyzed, and based on a preset stability threshold... θ Determine the target number of consecutive stable frames at that frequency point.

[0069] For example, the target number of consecutive stable frames. It can be determined in the following way: ; in, The target number of consecutive stable frames, i.e., the number of frames... n The number of consecutive stable frames corresponding to a frame. For the first n -1 frame corresponds to the number of consecutive stable frames. For the first n The change in spectral value of the frame. θ The preset stability threshold θ .

[0070] Based on a preset stability threshold θ For each audio frame in a plurality of audio frames, based on the change in spectral value of that audio frame relative to the previous frame. This determines the spectral stability of a frequency point within that audio frame. If the spectral change is less than a stability threshold... θ If the frequency point is stable within that audio frame, then the number of consecutive stable frames is incremented, for example, by 1; if the spectral change is greater than or equal to the stability threshold... θ If the frequency point is unstable or active in that audio frame, the recorded number of consecutive stable frames is reset, for example, to 1. Thus, when the current audio frame is the last audio frame in a series of consecutive audio frames, the target number of consecutive stable frames can be derived from the recorded number of consecutive stable frames corresponding to the current frame. Among them, the number of consecutive stable frames of the target. It can be used as an indicator of the stability of the spectral value at that frequency point and to determine the fusion weight of historical frames.

[0071] In some alternative implementations, a target number of consecutive stable frames can be set. The maximum value, i.e., the maximum number of consecutive stable frames. This is to avoid excessively long historical observation information. The specific values ​​can be flexibly set according to actual application needs.

[0072] Step 2023: Determine the historical frame fusion weight for this frequency point based on the target number of consecutive stable frames.

[0073] In some optional implementations, dynamic historical frame fusion weights can be constructed for each frequency point using a preset linear or nonlinear mapping function based on the statistical target number of consecutive stable frames. For example, the value of the historical frame fusion weight can be in the range of [0, 1]. When the spectral stability of a frequency point is high, the weight value is close to 1, indicating a large contribution from historical frames to that frequency point, which is beneficial for noise suppression; when the spectral stability of a frequency point is low, the weight value is close to 0, indicating a large contribution from the current frame to that frequency point, which is beneficial for preserving more speech details.

[0074] For example, it can be based on the target number of consecutive stable frames. The historical frame fusion weights for a given frequency point are determined by setting a threshold for the number of consecutive stable frames. Specifically, the historical frame fusion weights for that frequency point can be determined as follows: : ; in, For frequency point Historical frame fusion weights, This is the set threshold for the number of consecutive stable frames, i.e., the maximum number of consecutive stable frames.

[0075] In addition, a maximum value can be set for the historical frame fusion weight. For example, the maximum value can be set to 0.6, 0.7, or 0.8. If the determined historical frame fusion weight exceeds the set maximum value, the value of the historical frame fusion weight will be set to the set maximum value. This avoids the current frame being overly smoothed or weakened, ensuring that the speech details in the current frame are not lost excessively.

[0076] Furthermore, according to some other exemplary implementations, the historical frame fusion weight can also be determined based on a preset stable frame number threshold M, according to the target consecutive stable frame number, where the value of M can be a positive integer greater than 1. For example, if the target consecutive stable frame number... If the frequency is greater than or equal to M, then the frequency point can be identified as a stable frequency point, and its stability is considered to be relatively high. Otherwise, if the target has a continuous stable frame count... If the value is less than M, the frequency point is identified as an active frequency point, indicating low stability. Furthermore, for stable frequency points, the historical frame fusion weight can be set within a first weight range; for active frequency points, the historical frame fusion weight can be set within a second weight range, where the upper limit of the second weight range can be less than the lower limit of the first weight range. For example, the first weight range can be within the range [0.5, 1], and the second weight range can be within the range [0, 0.5), etc.

[0077] Based on the above embodiments, and according to some exemplary implementations of this disclosure, the original spectral value of each frequency point in the current frame can be fused with the historical spectral value in the historical frames based on the determined historical frame fusion weights to obtain the fused spectral value of that frequency point in the current frame.

[0078] In some optional implementations, the fusion weight of the current frame can be determined based on the difference between the preset fusion weight and the fusion weight of the historical frames. Then, based on the fusion weight of the current frame and the fusion weight of the historical frames, the original spectral value of each frequency point in the current frame is weighted and fused with the fused spectral value in the previous audio frame to obtain the fused spectral value of that frequency point in the current frame. The preset fusion weight can be 1 or other values.

[0079] In practical applications, for audio signal superposition or frequency domain fusion scenarios, directly adding two amplitude or power spectrum values ​​(i.e., energy values) may cause clipping or amplitude / energy overflow. By setting the preset fusion weight to 1, ensuring that the sum of the two weights is always 1 for weighted fusion, it can be guaranteed that the total amplitude or energy after fusion will not differ too much from the original spectrum value, thus maintaining the consistency of the signal during audio processing.

[0080] For example, the fusion weights can be determined based on the historical frames. The original spectral value of that frequency point in the current frame The fused spectral value of the previous frame Weighted fusion is performed, and the fused spectrum value of this frequency point is obtained as follows: : ; in, For historical frame fusion weights, This represents the fused spectrum value of that frequency point in the current frame. This is the fused spectral value of this frequency point in the previous frame of the current frame. This is the original spectral value of that frequency point in the current frame.

[0081] Historical frame fusion weights The value of is located in [0, 1], and the fusion weight of the current frame is . Therefore, the fusion weight of the current frame is also located in [0, 1], and the sum of the two is 1.

[0082] Thus, based on the fused spectral value of that frequency point in the historical frame of the previous moment... The signal is weighted and fused with the original spectral value in the current frame, and the weighting factor can be dynamically adjusted according to the signal characteristics, thereby effectively suppressing high-frequency noise components. At the same time, since the historical spectral value information is fused into the output signal of the current frame, the fused output signal is more continuous and smooth, and even if there are large fluctuations in the current frame, it will not cause drastic jumps.

[0083] Based on the audio processing method described above, a fusion scheme is proposed that determines the historical frame fusion weight based on the statistical target number of continuous stable frames, and performs weighted fusion of the current frame spectral value and the historical frame spectral value based on the historical frame fusion weight. The historical frame fusion weight is dynamically determined by statistically analyzing the spectral value changes of each frequency point across multiple consecutive audio frames. Based on the determined historical frame fusion weight, the original spectral value of the current frame is fused with the historical spectral value of the historical frame. This allows noise suppression to combine the long-term characteristics of frequency points across time, achieving enhanced suppression of stable noise components, while lightly processing speech points to reduce speech distortion and maintain the transient characteristics and clarity of speech.

[0084] According to the embodiments of this disclosure, after obtaining the fused spectrum value of each frequency point in the current frame, a denoised spectrum for each audio frame can be generated based on the fused spectrum values ​​of all frequency points in each original spectrum. This denoised spectrum is the spectrum after denoising processing via frequency domain fusion. Through dynamic frequency domain fusion, the amplitude or power of the noise frequency components is suppressed, thereby improving the denoising effect while preserving speech clarity and naturalness.

[0085] It should be understood that, without departing from the teachings of this disclosure, the stability of the continuous spectral value at each frequency point can also be assessed using statistical information other than the target number of continuously stable frames mentioned above. For example, the stability of the frequency point's spectral value can be judged by observing the average or rate of change of the original spectral values ​​of multiple consecutive historical frames, and this disclosure does not impose any restrictions on this. In addition, without departing from the teachings of this disclosure, the historical frame fusion weights can also be determined using other linear or nonlinear mapping functions based on statistical information representing temporal continuity to adapt to the audio noise reduction requirements of different scenarios, and this disclosure also does not impose any restrictions on this.

[0086] It should also be understood that the steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. The dependencies, execution order and combination of steps are also unrestricted. Adding insignificant modifications or introducing insignificant designs to the process, as long as they do not significantly change the core logical relationship of the process, are all within the protection scope of this disclosure.

[0087] Furthermore, based on the above embodiments and according to some exemplary implementations of this disclosure, the target audio signal can be generated based on the noise reduction spectrum of each audio frame in the following manner: First, perform an inverse Fourier transform on the noise reduction spectrum of each audio frame to convert it into a time-domain noise reduction signal; then, perform windowing and overlapping addition processing on the time-domain noise reduction signal to generate the target audio signal.

[0088] According to some embodiments of this disclosure, based on the fused spectral value of each frequency point in each original spectrum... Then, based on the fused spectral values ​​of all frequency points in the audio frame, the denoised spectrum of the audio frame is obtained, i.e., the denoised frequency domain representation. An inverse fast fourier transform (IFFT) is then performed on the denoised spectrum of each audio frame to restore it to the time-domain denoised signal. As shown below: ; in, This represents the noise reduction spectrum of the audio frame. This is the time-domain denoised signal of the audio frame, i.e., the enhanced time-domain frame after denoising. IFFT stands for Inverse Fourier Transform.

[0089] By denoising the signal in the time domain Windowing and overlapping processing are performed to generate the target audio signal.

[0090] If there is overlap between the enhanced temporal frames, a windowing and overlap summation method is used to reconstruct a continuous temporal audio signal, as follows: ; in, For the target audio signal, t For time, For frame shift, For window functions.

[0091] Windowing can eliminate discontinuities between frames, and then by overlapping and adding the discrete time-domain frames in chronological order, a continuous time-domain signal can be obtained. ( ), which is the target audio signal after noise reduction processing.

[0092] According to the above implementation method, after performing inverse Fourier transform to convert the frequency domain frame into a time domain frame, each frame is multiplied by a window function to smooth the frame edges to meet the reconstruction conditions and ensure that overlapping frames are smoothly superimposed. Then, all time domain frames are superimposed and added together to form a continuous time domain signal, thereby realizing lossless reconstruction from the noise-reduced signal after dynamic frequency domain fusion processing to a high-quality time domain signal, while preserving the details and natural listening experience of the original audio signal.

[0093] The audio processing method provided in this disclosure will be illustrated below with specific application scenarios.

[0094] See Figure 3 This illustrates an exemplary flow of an audio processing method in a specific application scenario. In this application scenario, the audio processing flow 300 can be divided into the following steps: Step 301, signal preprocessing.

[0095] In this example, the acquired audio signal is a single-channel speech with a sampling rate of 16 kHz (kilohertz).

[0096] In this step, the acquired audio signal is preprocessed.

[0097] First, the audio signal is normalized to scale its amplitude range to [-1, 1]. The purpose of normalization is to prevent overflow caused by excessive amplitude in subsequent FFT calculations.

[0098] Then, the audio signal is divided into overlapping short-time frames through short-time segmentation, and windowing is applied. The specific parameters can be set as follows: Frame length =512 points, approximately 32ms.

[0099] Frame shift =256 points, 50% overlap, which means the corresponding frame shift is half the frame length.

[0100] The window function can be a Hamming window, as shown below: ; in, w ( t ) is a window function. The frame length.

[0101] Each frame is processed by a Hamming window to obtain a continuous windowed frame sequence, which is used for subsequent frequency domain analysis calculations.

[0102] It should be understood that the analysis window type, frame length, and frame shift can be flexibly set according to the noise scenario of the actual application, and this disclosure does not impose specific restrictions.

[0103] Step 302: Convert to frequency domain signal.

[0104] Perform an FFT transform on each audio frame in the frame sequence to obtain the frequency domain representation of each frame. Each frame's frequency domain representation includes multiple specific frequency components, or frequency points, and the corresponding signal spectrum for each frequency point. This can be understood as the horizontal axis of the frequency domain representation representing all frequency points, and the vertical axis representing the amplitude or power spectrum value corresponding to each frequency point.

[0105] Frequency resolution, i.e., the spacing between frequency points, is determined by the preset sampling rate and the number of FFT points. The number of FFT points can be related to the frame length. L Consistent. For example, when the FFT point count is 512, the frequency resolution is 16kHz / 512=31.25Hz.

[0106] When the number of FFT points is consistent with the frame length, each sampling point of the time-domain signal in each frame can be directly mapped to a frequency point in the frequency domain, ensuring a direct correspondence between the time and frequency domains. No additional interpolation or zero-padding operations are required, thus avoiding redundant calculations.

[0107] Step 303: Determine the continuous stability of the frequency points.

[0108] Frequency domain representation for each frame Each frequency point Determine whether the following conditions are met: ; in, Frequency The original spectral values ​​in the current frame. Frequency The original spectral value in the previous frame of the current frame.

[0109] In this example, frequency point The spectral change value in each audio frame is the difference in power spectral values ​​of that frequency point in two adjacent audio frames. If the spectral change value of that frequency point in the current frame is less than a preset stability threshold... Then the frequency point is considered It is stable in the current frame.

[0110] For example, stability threshold =2dB. dB is used as the unit here because the human ear's perception of sound is logarithmic, not linear. A 2dB change in speech energy can be considered a slight fluctuation. It should be understood that the stability threshold... The value can be set according to actual needs.

[0111] Furthermore, it can be based on frequency points The target number of consecutive stable frames for a given frequency point is determined by the spectral variation values ​​across multiple consecutive audio frames. As shown in Table 1 below.

[0112] Table 1

[0113] In practical applications, the spectral value change can be counted starting from the second frame in the frame sequence, while the spectral value change of the first frame can be considered to be 0.

[0114] Step 304: Determine the historical frame fusion weights.

[0115] Specifically, based on the target number of consecutive stable frames Dynamically determine each frequency point Historical frame fusion weight If a certain frequency point is stable across multiple consecutive audio frames, it indicates that the frequency point is currently in a state of small energy change and may belong to background noise. In this case, a higher historical frame fusion weight can be set for it.

[0116] In this example, to avoid excessively long historical data, a maximum number of consecutive stable frames can be set. =6, with 6 Historical frames within a time period of 16ms ≈ 96ms are used as the observation interval.

[0117] Target continuous stable frame count The more historical frames, the higher the fusion weight. The larger the weight, the greater the weight of historical frame fusion. The numerical range can be between [0,1]. Historical frame fusion weights For specific settings, please refer to the relevant instructions in step 2023 above, which will not be repeated here.

[0118] Step 305, dynamic frequency domain fusion.

[0119] Based on the determined historical frame fusion weights The original spectral value of the frequency point in the current frame is weighted and fused with the fused spectral value in the historical frames to obtain the fused spectral value of the frequency point in the current frame. Based on the fused spectral values ​​of all frequency points in each original spectrum, a noise-reduced spectrum for each audio frame is generated. For details regarding the specific method of dynamic frequency domain fusion, please refer to the relevant descriptions in the foregoing embodiments; they will not be repeated here.

[0120] Step 306: Restore to time domain signal.

[0121] Perform an IFFT transform on the noise reduction spectrum to obtain the noise suppression time domain segment for each frame.

[0122] Step 307: Generate the target audio signal.

[0123] Finally, use the frame shift consistent with preprocessing. R Combined with window functions, i.e. R =256, using the Hamming window, to obtain the final high-quality noise-reduced frequency, i.e., the target audio signal.

[0124] The specific processing procedures for steps 306 and 307 can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0125] According to the audio processing method of the above-described embodiments of this disclosure, the spectral stability of each frequency point in the time dimension is statistically analyzed, and an adaptive noise reduction weight is set based on the statistical results. Through this frequency-point level dynamic frequency domain fusion, refined noise reduction processing can be achieved in different time periods and different frequency ranges, making the algorithm more intelligent and controllable. After dynamic frequency domain fusion, the target audio signal output will have noise suppressed, while retaining speech clarity and naturalness, thereby improving the overall noise reduction effect.

[0126] Compared to current mainstream noise reduction solutions, the noise suppression method based on dynamic frequency domain fusion disclosed above does not require the construction of a noise estimation model or a deep learning-based neural network model. Therefore, it has lower computational complexity, lower deployment requirements, and stronger adaptability, improving the accuracy of audio noise reduction and making it well-suited for low-power chips. It also avoids the accumulation of noise estimation errors, resulting in more uniform and less abrupt noise volume over time, and a more stable and natural-sounding output audio signal. Furthermore, parameters such as the window size, number of fusion frames, stability threshold, and historical frame fusion weights can be flexibly configured and adjusted to adapt to the CPU / memory capabilities of different devices and the noise reduction application requirements. This allows for weight calculation and fusion strategies for different application scenarios, such as more historical frames, finer-grained threshold control, or reducing the number of fusion frames to lower computational complexity, thereby achieving a controllable balance between noise reduction and computational load.

[0127] The audio processing methods described above illustrate from various aspects how to process input audio signals to obtain target audio signals. Furthermore, as an implementation of the methods shown in the figures above, this disclosure provides an embodiment of an audio processing apparatus, which corresponds to the aforementioned audio processing method embodiments and can be specifically applied to various electronic devices.

[0128] like Figure 4 As shown, the audio processing device 400 of this embodiment includes: a signal processing module 410, a weight determination module 420, a spectrum fusion module 430, a spectrum generation module 440, and a signal generation module 450.

[0129] In this embodiment, the signal processing module 410 is configured to acquire the original spectrum of each audio frame in the frame sequence of the audio signal. The weight determination module 420 is configured to determine the historical frame fusion weight of any frequency point in each original spectrum based on the original spectral values ​​of the frequency point in multiple consecutive audio frames. The multiple audio frames include the current frame and at least one corresponding historical frame; the historical frame fusion weight is related to the stability of the spectral values ​​of the frequency point in multiple audio frames. The spectral value fusion module 430 is configured to fuse the original spectral value of the frequency point in the current frame with the historical spectral value in the historical frame based on the historical frame fusion weight to obtain the fused spectral value of the frequency point in the current frame. The spectrum generation module 440 is configured to generate the denoised spectrum of each audio frame based on the fused spectral values ​​of all frequency points in each original spectrum. The signal generation module 450 is configured to generate the target audio signal based on the denoised spectrum of each audio frame.

[0130] In some optional implementations of this embodiment, the weight determination module 420 is further configured to: for each audio frame in a plurality of audio frames, determine the spectral value change value of the frequency point in the audio frame based on the original spectral value of the frequency point in the audio frame and the original spectral value in the previous audio frame of the audio frame; determine the target number of consecutive stable frames based on the spectral value change value of the frequency point in each audio frame in a plurality of audio frames and a preset stability threshold; and determine the historical frame fusion weight of the frequency point based on the target number of consecutive stable frames.

[0131] In some optional implementations of this embodiment, the weight determination module 420 is further configured to: for each audio frame in a plurality of audio frames, if the spectral value change value of the audio frame is less than the stability threshold, increment the number of recorded consecutive stable frames by 1; if the spectral value change value of the audio frame is greater than or equal to the stability threshold, reset the number of recorded consecutive stable frames; and if the audio frame is the current frame, determine the number of recorded consecutive stable frames corresponding to the current frame as the target number of consecutive stable frames.

[0132] In some optional implementations of this embodiment, the weight determination module 420 is further configured to: determine the historical frame fusion weight based on the target number of consecutive stable frames and the threshold number of consecutive stable frames.

[0133] In some optional implementations of this embodiment, the weight determination module 420 is further configured to: set the value of the historical frame fusion weight in a first weight range in response to the target consecutive stable frame number being greater than or equal to a preset stable frame number threshold; and set the value of the historical frame fusion weight in a second weight range in response to the target consecutive stable frame number being less than the preset stable frame number threshold; wherein the maximum value of the second weight range is less than the minimum value of the first weight range.

[0134] In some optional implementations of this embodiment, the spectral fusion module 430 is further configured to: determine the fusion weight of the current frame based on the difference between the preset fusion weight and the fusion weight of the historical frame; and, based on the fusion weight of the current frame and the fusion weight of the historical frame, perform weighted fusion of the original spectral value of the frequency point in the current frame with the fusion spectral value in the previous audio frame of the current frame to obtain the fusion spectral value of the frequency point in the current frame.

[0135] In some optional implementations of this embodiment, the signal processing module 410 is further configured to: perform short-time framing and windowing processing on the audio signal to obtain a frame sequence; perform Fourier transform on each audio frame in the frame sequence to obtain the frequency domain representation of each audio frame; and obtain the original spectrum of each audio frame based on the frequency domain representation.

[0136] In some optional implementations of this embodiment, the signal generation module 450 is further configured to: perform inverse Fourier transform on the noise reduction spectrum of each audio frame to convert it into a time-domain noise reduction signal; and perform windowing and overlapping addition processing on the time-domain noise reduction signal to generate the target audio signal.

[0137] In this embodiment, more specific processing and effects of the signal processing module 410, weight determination module 420, spectral fusion module 430, spectrum generation module 440, and signal generation module 450 of the audio processing device 400 can be found in [reference needed]. Figure 2 or Figure 3 The relevant descriptions in the aforementioned method embodiments are not repeated here.

[0138] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0139] According to embodiments of this disclosure, this disclosure also provides an electronic device and a computer-readable storage medium.

[0140] An electronic device includes: one or more processors; and a storage device having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200 or 300.

[0141] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in process 200 or 300.

[0142] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0143] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0144] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0145] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as audio processing methods. For example, in some embodiments, the audio processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the audio processing method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform audio processing methods by any other suitable means (e.g., by means of firmware).

[0146] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. This disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Embodiments of this disclosure can be implemented, in whole or in part, by software, hardware, firmware, or any other combination. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. This application can take the form of a computer program product embodied on one or more computer-readable storage media containing computer-usable program code. A computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the flow or function according to embodiments of this disclosure is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0150] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An audio processing method, characterized by, include: Obtain the raw spectrum of each audio frame in the frame sequence of the audio signal; For any frequency point in each of the original spectra, the historical frame fusion weight of the frequency point is determined based on the original spectral values ​​of the frequency point in multiple consecutive audio frames; wherein, the multiple audio frames include the current frame and at least one historical frame corresponding to the current frame; the historical frame fusion weight is related to the stability of the spectral values ​​of the frequency point in the multiple audio frames; Based on the historical frame fusion weight, the original spectral value of the frequency point in the current frame is fused with the historical spectral value in the historical frame to obtain the fused spectral value of the frequency point in the current frame; Based on the fused spectral values ​​of all frequency points in each of the original spectra, the noise-reduced spectrum of each audio frame is generated; The target audio signal is generated based on the noise reduction spectrum of each audio frame.

2. The method according to claim 1, characterized in that, Determining the historical frame fusion weight of the frequency point based on its original spectral values ​​in multiple consecutive audio frames includes: For each of the plurality of audio frames, the change in the spectral value of the frequency point in the audio frame is determined based on the original spectral value of the frequency point in the audio frame and the original spectral value in the previous audio frame. The target number of consecutive stable frames is determined based on the spectral value change of each audio frame in the plurality of audio frames at the frequency point and a preset stability threshold. The historical frame fusion weight of the frequency point is determined based on the target number of consecutive stable frames.

3. The method of claim 2, wherein, The step of determining the target number of consecutive stable frames based on the spectral change value of each audio frame in the plurality of audio frames at the frequency point and a preset stability threshold includes: For each of the plurality of audio frames, if the spectral value change of the audio frame is less than the stability threshold, the number of recorded consecutive stable frames is incremented by 1. If the spectral value change of the audio frame is greater than or equal to the stability threshold, the number of consecutive stable frames recorded will be reset. When the audio frame is the current frame, the number of recorded consecutive stable frames corresponding to the current frame is determined as the target number of consecutive stable frames.

4. The method of claim 2, wherein, The step of determining the historical frame fusion weight of the frequency point based on the target continuous stable frame number includes: The historical frame fusion weights are determined based on the target number of consecutive stable frames and the threshold number of consecutive stable frames.

5. The method of claim 2, wherein, The step of determining the historical frame fusion weight of the frequency point based on the target continuous stable frame number includes: In response to the target consecutive stable frame number being greater than or equal to a preset stable frame number threshold, the value of the historical frame fusion weight is set to a first weight range; In response to the target number of consecutive stable frames being less than the preset number of stable frames threshold, the value of the historical frame fusion weight is set to a second weight range; Wherein, the upper limit of the weight of the second weight range is less than the lower limit of the weight of the first weight range.

6. The method according to any one of claims 1-5, characterized in that, The step of fusing the original spectral value of the frequency point in the current frame with the historical spectral value in the historical frames based on the historical frame fusion weight to obtain the fused spectral value of the frequency point in the current frame includes: The fusion weight of the current frame is determined based on the difference between the preset fusion weight and the fusion weight of the historical frames. Based on the fusion weight of the current frame and the fusion weight of the historical frames, the original spectral value of the frequency point in the current frame is weighted and fused with the fused spectral value in the previous audio frame of the current frame to obtain the fused spectral value of the frequency point in the current frame.

7. The method of claim 1, wherein, The acquisition of the original spectrum of each audio frame in the frame sequence of the audio signal includes: The audio signal is subjected to short-time framing and windowing processing to obtain the frame sequence; Perform a Fourier transform on each audio frame in the frame sequence to obtain the frequency domain representation of each audio frame; The original spectrum of each audio frame is obtained based on the frequency domain representation.

8. The method of claim 7, wherein, The total number of frequency points included in each of the original spectra is determined based on a preset sampling rate and Fourier points.

9. The method according to claim 7 or 8, characterized in that, The generation of the target audio signal based on the noise reduction spectrum of each audio frame includes: The inverse Fourier transform is performed on the noise reduction spectrum of each audio frame to convert it into a time-domain noise reduction signal; The time-domain noise-reduced signal is subjected to windowing and overlapping addition processing to generate the target audio signal.

10. An audio processing device, characterized by include: The signal processing module is configured to acquire the raw spectrum of each audio frame in the frame sequence of the audio signal; The weight determination module is configured to, for any frequency point in each of the original spectra, determine the historical frame fusion weight of the frequency point based on the original spectral values ​​of the frequency point in a plurality of consecutive audio frames; wherein, the plurality of audio frames includes the current frame and at least one historical frame corresponding to the current frame; the historical frame fusion weight is related to the stability of the spectral values ​​of the frequency point in the plurality of audio frames; The spectral fusion module is configured to fuse the original spectral value of the frequency point in the current frame with the historical spectral value in the historical frame based on the historical frame fusion weight, so as to obtain the fused spectral value of the frequency point in the current frame. The spectrum generation module is configured to generate the noise-reduced spectrum of each audio frame based on the fused spectrum values ​​of all frequency points in each original spectrum; The signal generation module is configured to generate a target audio signal based on the noise reduction spectrum of each audio frame.

11. An electronic device, comprising: include: One or more processors; Storage device, on which one or more computer programs are stored, When one or more computer programs are executed by one or more processors, the one or more processors implement the audio processing method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When a computer program is executed by a processor, it implements the audio processing method as described in any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When executed by a processor, a computer program implements the audio processing method as described in any one of claims 1-9.

Images