Audio processing method and related device

Abstract

The invention discloses an audio processing method and a related device, and relates to the technical field of audio processing, and the method comprises the steps: obtaining a current audio frame; determining the loudness of the current audio frame and the average peak loudness of the current frame; determining a difference value between the first target loudness and the loudness of the current audio frame to obtain a first difference value, determining a difference value between the second target loudness and the average peak loudness of the current frame to obtain a second difference value, and taking the smaller value of the first difference value and the second difference value as a target loudness gain value; and performing audio enhancement processing on the current audio frame based on the target loudness gain value to obtain the processed current audio frame. According to the invention, each frame of audio in the audio stream can be subjected to audio enhancement processing, streaming data processing is realized, and real-time frame processing of a player can be adapted.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio processing method and related apparatus. Background Technology

[0002] In digital signal workstations, low-frequency enhancement is used to enhance the energy of low-frequency audio (i.e., audio with a frequency lower than a preset frequency, which can be 250Hz). Current low-frequency enhancement methods usually process the entire audio segment before outputting the signal. This method is not a streaming method that can be used by players. Therefore, how to make the low-frequency enhancement method adapt to the real-time frame processing of players through streaming data processing is an urgent problem to be solved. Summary of the Invention

[0003] In view of the above problems, this application provides an audio processing method and related apparatus, which can adapt to the real-time frame processing of the player by streaming data. The specific solution is as follows:

[0004] The first aspect of this application provides an audio processing method, including:

[0005] Obtain the current audio frame; wherein, the current audio frame includes sampling data of multiple audio sampling points;

[0006] Determine the loudness of the current audio frame and the average peak loudness of the current frame; wherein, the average peak loudness of the current frame is the average of the peak loudness of the sampled data of all audio sampling points of the current audio frame;

[0007] The difference between the first target loudness and the loudness of the current audio frame is determined to obtain the first difference value. The difference between the second target loudness and the average peak loudness of the current frame is determined to obtain the second difference value. The smaller value between the first difference value and the second difference value is taken as the target loudness gain value.

[0008] Based on the target loudness gain value, the current audio frame is subjected to audio enhancement processing to obtain the processed current audio frame.

[0009] In one possible implementation, after taking the smaller of the first difference and the second difference as the target loudness gain value, the method further includes:

[0010] The current audio frame is filtered to obtain a target frequency signal; wherein the target frequency signal is an audio signal with a frequency lower than a preset frequency.

[0011] Determine the proportion of the target frequency signal in the current audio frame;

[0012] When the proportion is less than the proportion threshold, an audio enhancement ratio coefficient is determined based on the proportion, and the product of the audio enhancement ratio coefficient and the target loudness gain value is used as the first loudness gain value.

[0013] When the proportion is not less than the proportion threshold, the proportion is used as the weight of the second difference, the difference between 1 and the proportion is used as the weight of the first difference, the product of the first difference and the weight of the first difference is added to the product of the second difference and the weight of the second difference to obtain a summation value, and half of the summation value is used as the second loudness gain value.

[0014] The step of performing audio enhancement processing on the current audio frame based on the target loudness gain value to obtain the processed current audio frame includes:

[0015] When the proportion is less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the first loudness gain value to obtain the processed current audio frame.

[0016] If the proportion is not less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the second loudness gain value to obtain the processed current audio frame.

[0017] In one possible implementation, determining the audio enhancement ratio coefficient based on the ratio includes:

[0018] If the percentage is less than the target value, then the audio enhancement ratio coefficient is determined to be 1;

[0019] If the percentage is not less than the target value and the percentage is less than the percentage threshold, then the audio enhancement ratio coefficient is determined to be a value greater than 0 and less than 1.

[0020] If the percentage is not less than the percentage threshold, then the audio enhancement ratio coefficient is determined to be 0.

[0021] In one possible implementation, after obtaining the first loudness gain value or the second loudness gain value, the method further includes:

[0022] Determine the average loudness and average peak loudness of audio frames in a sliding window; wherein, the audio frames in the sliding window include the current audio frame and p audio frames preceding the current audio frame, the average loudness of audio frames is the average loudness of audio frames in the sliding window, and the average peak loudness is the average peak loudness of audio frames in the sliding window;

[0023] The difference between the average peak loudness of the current frame and the average peak loudness is determined to obtain a third difference value; the difference between the average loudness of the audio frame and the average peak loudness is determined to obtain a fourth difference value.

[0024] If the absolute value of the third difference is greater than the mutation threshold and the absolute value of the fourth difference is greater than the mutation threshold, the sum of the first loudness gain value and the gain adjustment value is taken as the adjusted first loudness gain value, and the sum of the second loudness gain value and the gain adjustment value is taken as the adjusted second loudness gain value; wherein, when both the third difference and the fourth difference are positive, the gain adjustment value is positive, and when both the third difference and the fourth difference are negative, the gain adjustment value is negative.

[0025] When the proportion is less than the proportion threshold, the audio enhancement processing of the current audio frame is performed based on the first loudness gain value to obtain the processed current audio frame, including:

[0026] When the proportion is less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the adjusted first loudness gain value to obtain the processed current audio frame.

[0027] When the proportion is not less than the proportion threshold, the audio enhancement processing of the current audio frame is performed based on the second loudness gain value to obtain the processed current audio frame, including:

[0028] If the proportion is not less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the adjusted second loudness gain value to obtain the processed current audio frame.

[0029] In one possible implementation, the audio enhancement processing of the current audio frame based on the adjusted first loudness gain value to obtain the processed current audio frame includes:

[0030] Obtain the parameter coefficients of the delay line; wherein, the parameter coefficients of the delay line include the sampling data and output data of historical audio sampling points;

[0031] The filter coefficients are determined based on the adjusted first loudness gain value;

[0032] Based on the parameter coefficients of the delay line, the filter coefficients, and the sampling data of the current audio sampling point of the current audio frame, the output data of the current audio sampling point of the current audio frame is determined;

[0033] Determine whether the current audio sampling point of the current audio frame is the last audio sampling point of the current audio frame;

[0034] If the current audio sampling point of the current audio frame is not the last audio sampling point of the current audio frame, then the parameter coefficients of the delay line are updated based on the output data of the current audio sampling point of the current audio frame to obtain new parameter coefficients of the delay line. The next audio sampling point of the current audio frame is taken as the current audio sampling point of the current audio frame. Based on the new parameter coefficients of the delay line, the filter coefficients, and the sampling data of the current audio sampling point of the current audio frame, the output data of the current audio sampling point of the current audio frame is determined. Then, the step of determining whether the current audio sampling point of the current audio frame is the last audio sampling point of the current audio frame is executed.

[0035] If the current audio sampling point of the current audio frame is the last audio sampling point of the current audio frame, then the output data of all audio sampling points of the current audio frame is used as the processed current audio frame.

[0036] In one possible implementation, following the processed current audio frame, the following is also included:

[0037] When the adjusted first loudness gain value or the adjusted second loudness gain value is updated, the sampled data of the last audio sample point of the current audio frame is used as the first parameter coefficient of the delay line, the sampled data of the second-to-last audio sample point of the current audio frame is used as the second parameter coefficient of the delay line, the output data of the last audio sample point of the current audio frame is used as the third parameter coefficient of the delay line, and the output data of the second-to-last audio sample point of the current audio frame is used as the fourth parameter coefficient of the delay line.

[0038] In one possible implementation, following the processed current audio frame, the following is also included:

[0039] If the current audio frame is the first audio frame, then N zeros are used as the output data of the first N audio sampling points of the current smoothed audio frame, and the output data of the first MN audio sampling points of the current audio frame are used as the output data of the last MN audio sampling points of the current smoothed audio frame, thus obtaining the current smoothed audio frame; where M is the total number of audio sampling points of the processed current audio frame, N is the preset number of audio sampling points, and N < M.

[0040] If the current audio frame is not the first audio frame, the output data of the last N audio sampling points of the previous audio frame is used as the first data, and the output data of the first N audio sampling points of the current audio frame is used as the second data. The average value of the first data is determined to obtain the first average value. The average value of the second data is determined to obtain the second average value. The difference between the first average value and the second average value is determined to obtain the target difference value. The product of the output data of each audio sampling point of the first data and the power operation value is determined to obtain the output data of the first N audio sampling points of the current smoothed audio frame. The output data of the first MN audio sampling points of the current audio frame is used as the output data of the last MN audio sampling points of the current smoothed audio frame to obtain the current smoothed audio frame. The power operation value is obtained by exponentiation with the target difference value multiplied by the target ratio value to the base 10. The target ratio value is the ratio of the position of the output data of the audio sampling points of the first data in the first data to the total number of data in the first data.

[0041] A second aspect of this application provides an audio processing system, comprising:

[0042] An acquisition module is used to acquire the current audio frame; wherein, the current audio frame includes sampling data of multiple audio sampling points;

[0043] The first determining module is used to determine the loudness of the current audio frame and the average peak loudness of the current frame; wherein the average peak loudness of the current frame is the average of the peak loudness of the sampled data of all audio sampling points of the current audio frame;

[0044] The second determining module is used to determine the difference between the first target loudness and the loudness of the current audio frame to obtain a first difference, determine the difference between the second target loudness and the average peak loudness of the current frame to obtain a second difference, and take the smaller value between the first difference and the second difference as the target loudness gain value.

[0045] The output module is used to perform audio enhancement processing on the current audio frame based on the target loudness gain value to obtain the processed current audio frame.

[0046] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the audio processing method of the first aspect or any implementation thereof.

[0047] A fourth aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0048] The memory is used to store computer programs;

[0049] The processor is used to execute the computer program so that the electronic device can implement the audio processing method of the first aspect or any implementation thereof.

[0050] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to perform the audio processing method described in the first aspect or any implementation thereof.

[0051] By employing the above technical solutions, the audio processing method and related apparatus provided in this application acquire a current audio frame, wherein the current audio frame includes sampling data from multiple audio sampling points; determine the loudness of the current audio frame and the average peak loudness of the current frame; determine the difference between a first target loudness and the loudness of the current audio frame to obtain a first difference; determine the difference between a second target loudness and the average peak loudness of the current frame to obtain a second difference; use the smaller value between the first difference and the second difference as a target loudness gain value; and perform audio enhancement processing on the current audio frame based on the target loudness gain value to obtain a processed current audio frame. This application can perform audio enhancement processing on each frame of audio in an audio stream, realize streaming data processing, and can adapt to the real-time frame processing of a player. Attached Figure Description

[0052] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0053] Figure 1 A flowchart of an audio processing method provided in this application;

[0054] Figure 2 A structural diagram of an audio processing system provided in this application;

[0055] Figure 3 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0056] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0057] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0058] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0059] Singing (hereinafter referred to as karaoke) using audio equipment or applications is a popular form of entertainment. Generally, karaoke requires a professional karaoke environment, but it can also be done through a media player app. In a media player app, enhancing the low-frequency atmosphere of a song is a quick way to attract users, as low-frequency enhancement can effectively improve the accompaniment's atmosphere. However, karaoke within a media player app suffers from limitations in source material production and the lack of sound effects, resulting in a lack of atmosphere that truly engages the user. To address this, one approach is to modify each song within the media asset environment to suit karaoke, essentially reproducing all content, which consumes significant storage resources. Another approach is to use low-frequency enhancement algorithms. Current low-frequency enhancement algorithms use a fixed gain coefficient to increase the loudness of low frequencies and cannot analyze loudness and high / low frequencies based on different segments to intelligently adjust the gain. Furthermore, this low-frequency enhancement algorithm applies to the entire audio segment or selected audio segments to enhance the low-frequency energy. Digital signal workstations like Audition typically process the entire audio segment before outputting the signal. This contradicts the logic of frame processing in a player and is not a suitable streaming algorithm for a player. Furthermore, low-frequency enhancement cannot dynamically adjust the input loudness value based on different low-frequency energies.

[0060] Based on this, this application provides an audio processing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an audio processing method provided in an embodiment of this application, such as... Figure 1 As shown in the embodiment of this application, an audio processing method may include steps 101 to 104, which are described in detail below.

[0061] Step 101: Obtain the current audio frame; wherein, the current audio frame includes sampling data of multiple audio sampling points.

[0062] The current audio frame can be obtained by the player after acquiring the stereo audio stream, parsing the stereo PCM (Pulse-Code Modulation) data, and performing frame-level data reading. The resulting current audio frame is denoted as [example frame name]. It can also record the sampling frequency of the audio stream. The audio frames in most media players are in AAC (Advanced Audio Coding) format. AAC is a highly efficient lossy audio compression format, and its sound quality is superior to MP3 at the same bitrate. 128kbps AAC sound quality is equivalent to 160kbps MP3, and AAC supports higher resolution audio parameters. An AAC frame has 1024 audio samples, and the current audio frame includes sample data from multiple audio samples, specifically 1024 audio samples. The inability to retrieve any more audio stream indicates that the stereo audio stream has been completely read.

[0063] Step 102: Determine the loudness of the current audio frame and the average peak loudness of the current frame; wherein, the average peak loudness of the current frame is the average of the peak loudness of the sampled data of all audio sampling points of the current audio frame.

[0064] The loudness of the current audio frame is 1 / 3 of the loudness of the current audio frame. Loudness (ITU R BS.1770 standard): The average peak loudness of the current frame is the average of the peak loudnesses of all audio samples in the current audio frame. The loudness of the current audio frame is denoted as... The average peak loudness of the current frame is denoted as .

[0065] Alternatively, within the current audio frame, the maximum amplitude point can be found by taking the maximum absolute value of all amplitudes in all audio frames, and the maximum peak loudness can be calculated, denoted as . Set the silent loudness threshold, denoted as . If the maximum peak loudness Less than the silent loudness threshold This indicates that we are currently in the white noise stage. At this point, the loudness is less than the range within which the human voice can perceive a clear signal. Therefore, low-frequency enhancement is unnecessary; that is, subsequent steps are not performed. Instead, the next audio frame is acquired, and the loudness of the next frame and the average peak loudness of the current frame are determined. Optionally, a silent loudness threshold can be used. It can be -60dB. If the maximum peak loudness... Not less than the silent loudness threshold Then proceed to step 103.

[0066] Step 103: Determine the difference between the first target loudness and the loudness of the current audio frame to obtain the first difference; determine the difference between the second target loudness and the average peak loudness of the current frame to obtain the second difference; and take the smaller value between the first difference and the second difference as the target loudness gain value.

[0067] The first target loudness is denoted as... The first target loudness can be set to -1 dB. The difference between the first target loudness and the loudness of the current audio frame is determined to obtain the first difference. The second target loudness is denoted as The second target loudness can also be set to -1 dB. The difference between the second target loudness and the average peak loudness of the current frame is then determined to obtain the second difference. Compare the first difference. The second difference The smaller of the first difference and the second difference is taken as the target loudness gain value. The target loudness gain value This can be the initial loudness gain value used for low-frequency enhancement.

[0068] Step 104: Perform audio enhancement processing on the current audio frame based on the target loudness gain value to obtain the processed current audio frame.

[0069] After obtaining the target loudness gain value, it can be used to enhance the low frequencies of the current audio frame, resulting in the processed current audio frame. After obtaining the processed current audio frame, the next audio frame of the audio stream can be obtained, and step 102 can be executed again until all audio frames of the audio stream have been processed, and the final stereo frame data for playback is output.

[0070] This application parses stereo data, obtains stereo frame data, dynamically outputs low-frequency loudness gain values, and uses the loudness gain values ​​for low-frequency enhancement. It can perform audio enhancement processing on each frame of audio in the audio stream, realize streaming data processing, and can be adapted to the real-time frame processing of the player.

[0071] In one possible implementation, after taking the smaller of the first difference and the second difference as the target loudness gain value, the following is also included:

[0072] The current audio frame is filtered to obtain the target frequency signal; wherein, the target frequency signal is an audio signal with a frequency lower than a preset frequency;

[0073] Determine the proportion of the target frequency signal in the current audio frame;

[0074] When the proportion is less than the proportion threshold, the audio enhancement ratio coefficient is determined based on the proportion, and the product of the audio enhancement ratio coefficient and the target loudness gain value is used as the first loudness gain value.

[0075] If the percentage is not less than the percentage threshold, the percentage is used as the weight of the second difference, the difference between 1 and the percentage is used as the weight of the first difference, the product of the first difference and the weight of the first difference is added to the product of the second difference and the weight of the second difference to obtain the sum value, and half of the sum value is used as the second loudness gain value.

[0076] When enhancing low frequencies, since the spectral distribution is different for each frame, if the low-frequency energy accounts for a large proportion of the total spectral energy, and a relatively large target loudness gain value is applied... Enhancing low frequencies can cause percussion instruments to sound muffled. Therefore, this application addresses this issue by increasing the loudness of the entire spectrum when the low frequencies are already sufficiently strong and their energy constitutes a large proportion of the total spectral energy. Specifically, it calculates a second loudness gain value while ensuring the proportion is not less than a threshold. Conversely, when the low-frequency energy constitutes a small proportion of the total spectral energy, it determines the low-frequency loudness gain of the current audio frame, calculating a first loudness gain value while ensuring the proportion is less than a threshold.

[0077] The target frequency signal can be a low-frequency signal, which can be an audio signal with a frequency below 250Hz, and its proportion in the current audio frame is determined.

[0078] When filtering the current audio frame to obtain the target frequency signal, a low-pass filter can be used. This filter can be a Bassworth filter, set to second order, cutoff frequency to 250 fs, and filter type to low-pass filter. After obtaining the filter parameters, the current audio frame... Filtering is performed to obtain the target frequency signal. .

[0079] Then determine the proportion of the target frequency signal in the current audio frame. The calculation formula is as follows:

[0080]

[0081] In the formula, For target frequency signal The square of, For the target frequency signal Summing the squares of the terms, For the current audio frame The square of, For the current audio frame Sum of squares.

[0082] When the proportion is less than the proportion threshold, the audio enhancement ratio coefficient is determined based on the proportion, including:

[0083] If the proportion is less than the proportion threshold, and if the proportion is less than the target value, then the audio enhancement ratio coefficient is determined to be 1.

[0084] If the proportion is less than the proportion threshold, and the proportion is not less than the target value but less than the proportion threshold, then the audio enhancement ratio coefficient is determined to be a value greater than 0 and less than 1.

[0085] If the percentage is less than the percentage threshold, or if the percentage is not less than the percentage threshold, then the audio enhancement ratio coefficient is determined to be 0.

[0086] Target frequency signal percentage Determine the audio enhancement ratio coefficient Target frequency signal The smaller the proportion, the larger the audio enhancement ratio. The maximum audio enhancement ratio is 1, which represents the target frequency signal. The higher the proportion, the smaller the audio enhancement ratio coefficient, with a minimum value of 0. This proportion threshold can be 0.8 or other values, and the target value is less than the proportion threshold, which can be 0.4. Determine the audio enhancement ratio when it is less than 0.4. The value is 1, representing a percentage of The audio enhancement ratio coefficient is determined when it is not less than 0.4 and less than the proportion threshold of 0.8. For values ​​in the range (0,1), you can also consider the percentage. Size subdivision of multiple audio enhancement ratios For example, in 0.4≤ At 0.6, the audio enhancement ratio coefficient It can be 0.6 or other values, where 0.6 ≤ At 0.8, the audio enhancement ratio coefficient It can be 0.3 or other values, in the proportion. Audio enhancement ratio coefficient not less than 0.8 It is 0.

[0087] When the proportion is less than the proportion threshold, an audio enhancement ratio coefficient is determined based on the proportion. The product of the audio enhancement ratio coefficient and the target loudness gain value is taken as the first loudness gain value, which is denoted as […]. , for audio enhancement scaling factor With target loudness gain value The product of.

[0088] When the proportion is not less than the proportion threshold, that is, when When the value is 0, the loudness enhancement of the entire spectrum can be calculated, specifically the proportion. As the weight of the second difference, the difference between 1 and the proportion is used as the weight of the first difference. The product of the weights of the first difference and the second difference The product of the weights of the second difference and the summation is taken as the second loudness gain value. The calculation formula is as follows:

[0089]

[0090] In the formula, the second loudness gain value This represents the overall loudness boost value of the audio stream.

[0091] First difference The loudness is the difference between the target loudness and the loudness of the current audio frame, with a greater emphasis on the perception of mid-to-low frequency loudness. Therefore, it is assigned a value in the weighting. The second difference The difference between the second target loudness and the average peak loudness of the current frame is a full-band boost, not limited to low-frequency energy, and therefore assigned a weight. .

[0092] when When it is not 0, Recorded as 0, When it is 0, calculate This yields the final loudness of the entire frequency spectrum.

[0093] Optionally, the current audio frame is enhanced based on the target loudness gain value to obtain the processed current audio frame, including:

[0094] If the proportion is less than the proportion threshold, the current audio frame is enhanced based on the first loudness gain value to obtain the processed current audio frame.

[0095] If the proportion is not less than the proportion threshold, the current audio frame is enhanced based on the second loudness gain value to obtain the processed current audio frame.

[0096] In one possible implementation, after obtaining the first loudness gain value or the second loudness gain value, the following is also included:

[0097] Determine the average loudness and average peak loudness of the audio frames in the sliding window; wherein, the audio frames in the sliding window include the current audio frame and the p audio frames preceding the current audio frame, the average loudness of the audio frames is the average loudness of the audio frames in the sliding window, and the average peak loudness is the average peak loudness of the audio frames in the sliding window.

[0098] The difference between the average peak loudness of the current frame and the average peak loudness is determined to obtain the third difference; the difference between the average loudness of the audio frames and the average peak loudness is determined to obtain the fourth difference.

[0099] When the absolute value of the third difference is greater than the mutation threshold and the absolute value of the fourth difference is greater than the mutation threshold, the sum of the first loudness gain value and the gain adjustment value is taken as the adjusted first loudness gain value, and the sum of the second loudness gain value and the gain adjustment value is taken as the adjusted second loudness gain value; wherein, when both the third and fourth differences are positive, the gain adjustment value is positive, and when both the third and fourth differences are negative, the gain adjustment value is negative.

[0100] If the absolute value of the third difference is greater than the mutation threshold and the absolute value of the fourth difference is greater than the mutation threshold, the first loudness gain value and the second loudness gain value shall not be adjusted.

[0101] When determining the average loudness and average peak loudness of audio frames within a sliding window, two sliding windows can be set to judge loudness steps. Each sliding window can have a length of 3, and the two sliding windows can be denoted as follows: and Sliding window Record the loudness of the audio frame using LuFS Sliding window Record the average peak loudness of audio frames .

[0102] The first two audio frames entering the sliding window can be used to calculate the first loudness gain value in the manner described above. Or the second loudness gain value When the third frame of audio data arrives, the sliding window is already full. To prevent sudden changes in audio loudness, an average loudness calculation can be performed within the sliding window to determine the average loudness of the audio frames within the window. and average peak loudness The formula is as follows:

[0103]

[0104]

[0105] Determine the average peak loudness of the current frame Compared with the average peak loudness The difference is used to obtain the third difference, which determines the average loudness of the audio frames. Compared with the average peak loudness The difference is used to obtain the fourth difference. A mutation threshold is set, denoted as... The mutation threshold can be -6 dB or other values. If the absolute value of the third difference is greater than the mutation threshold and the absolute value of the fourth difference is greater than the mutation threshold... This indicates that the audio loudness is in a sudden change. Determine the gain adjustment value: when both the third and fourth differences are positive, the gain adjustment value is positive; when both the third and fourth differences are negative, the gain adjustment value is negative. If the absolute value of the gain adjustment value is 3dB, then when both the third and fourth differences are positive, the gain adjustment value is 3dB. Adjust the first loudness gain value... The sum of 3dB and 3dB is used as the adjusted first loudness gain value, and the second loudness gain value is... The second loudness gain value is adjusted to 3dB. When both the third and fourth differences are negative, the gain adjustment value is -3dB, and the first loudness gain value is adjusted accordingly. The sum of -3dB is used as the adjusted first loudness gain value, and the second loudness gain value is... The value after -3dB is used as the adjusted second loudness gain.

[0106] At this time, it is possible to... and The values ​​are stripped, and the retained data of the first frame of audio is stripped, then the sliding window... and Only the audio loudness data for the last two frames should exist. For example, [-1, -2, -3], Given [-1, -2, -3], the data after stripping is: [-2, -3, _], For [-2, -3, _]). If the current and and The absolute difference between the subtractions is not greater than 1 / 2. Then calculate and The subsequent frame return step "determines the average loudness and average peak loudness of the audio frames in the sliding window" until all audio frames have been processed.

[0107] In one possible implementation, provided the proportion is not less than a proportion threshold, the current audio frame is subjected to audio enhancement processing based on a second loudness gain value to obtain the processed current audio frame, including:

[0108] If the proportion is not less than the proportion threshold, the current audio frame is enhanced based on the adjusted second loudness gain value to obtain the processed current audio frame.

[0109] Optionally, the current audio frame is subjected to audio enhancement processing based on the adjusted second loudness gain value to obtain the processed current audio frame, including:

[0110] Using 10 as the base and the ratio of the adjusted second loudness gain value to 20 as the exponent, perform a power operation to obtain the loudness after linear gain conversion;

[0111] Multiply the loudness after linear gain conversion with the current audio frame to obtain the processed current audio frame.

[0112] When performing audio enhancement processing on the current audio frame based on the adjusted second loudness gain value, the second loudness gain value can be used as a reference. ,right To perform linear gain conversion, the loudness conversion is calculated sample by sample. When calculating the loudness conversion sample by sample, a power operation can be performed with the base 10 and the ratio of the adjusted second loudness gain value to 20 as the exponent, to obtain the loudness after linear gain conversion. The linear gain conversion formula is:

[0113]

[0114] Then, multiply the loudness after linear gain conversion by the current audio frame to obtain the processed current audio frame. That is, for the current audio frame x, multiply it bit-by-bit by each audio sample point in x to obtain the processed current audio frame y. The formula is: .

[0115] In one possible implementation, when the proportion is less than a proportion threshold, audio enhancement processing is performed on the current audio frame based on a first loudness gain value to obtain the processed current audio frame, including:

[0116] If the proportion is less than the proportion threshold, the current audio frame is enhanced based on the adjusted first loudness gain value to obtain the processed current audio frame.

[0117] Optionally, the current audio frame is subjected to audio enhancement processing based on the adjusted first loudness gain value to obtain the processed current audio frame, including:

[0118] Obtain the parameter coefficients of the delay line; where the parameter coefficients of the delay line include the sampling data and output data of historical audio sampling points;

[0119] The filter coefficients are determined based on the adjusted first loudness gain value;

[0120] Based on the parameter coefficients of the delay line, the filter coefficients, and the sampling data of the current audio sampling point of the current audio frame, the output data of the current audio sampling point of the current audio frame is determined.

[0121] Determine whether the current audio sample point of the current audio frame is the last audio sample point of the current audio frame;

[0122] If the current audio sampling point of the current audio frame is not the last audio sampling point of the current audio frame, update the parameter coefficients of the delay line based on the output data of the current audio sampling point of the current audio frame to obtain the new parameter coefficients of the delay line, and take the next audio sampling point of the current audio frame as the current audio sampling point of the current audio frame. Based on the new parameter coefficients of the delay line, the filter coefficients and the sampling data of the current audio sampling point of the current audio frame, determine the output data of the current audio sampling point of the current audio frame, and then perform a judgment on whether the current audio sampling point of the current audio frame is the last audio sampling point of the current audio frame.

[0123] If the current audio sample point of the current audio frame is the last audio sample point of the current audio frame, then the output data of all audio sample points of the current audio frame will be used as the processed current audio frame.

[0124] The filter coefficients include filter forward parameters and filter feedback parameters. The filter forward parameters include a first forward parameter, a second forward parameter, and a third forward parameter. The filter feedback parameters include a first feedback parameter, a second feedback parameter, and a third feedback parameter.

[0125] The parameter coefficients of the delay line include a first parameter coefficient, a second parameter coefficient, a third parameter coefficient, and a fourth parameter coefficient. The first parameter coefficient is the sampled data of the first audio sample point before the current audio sample point of the current audio frame. The second parameter coefficient is the sampled data of the second audio sample point before the current audio sample point of the current audio frame. The third parameter coefficient is the output data of the first audio sample point before the current audio sample point of the current audio frame. The fourth parameter coefficient is the output data of the second audio sample point before the current audio sample point of the current audio frame.

[0126] Optionally, based on the parameter coefficients of the delay line, the filter coefficients, and the sampling data of the current audio sampling point of the current audio frame, the output data of the current audio sampling point is determined, including:

[0127] Multiply the sampled data of the current audio sampling point of the current audio frame by the first positive parameter to obtain the first product;

[0128] Multiply the second positive parameter by the coefficient of the first parameter to obtain the second product;

[0129] Multiply the third positive parameter by the coefficient of the second parameter to obtain the third product;

[0130] Multiply the second feedback parameter by the coefficient of the third parameter to obtain the fourth product;

[0131] Multiply the third feedback parameter by the coefficient of the fourth parameter to obtain the fifth product;

[0132] Subtract the fourth product from the sum of the first, second, and third products, and then subtract the fifth product to obtain the target result;

[0133] The ratio of the target result to the first feedback parameter is used as the output data of the current audio sampling point of the current audio frame.

[0134] Specifically, if the current audio sampling point of the current audio frame is not the last audio sampling point of the current audio frame, the parameter coefficients of the delay line are updated based on the output data of the current audio sampling point of the current audio frame to obtain the new parameter coefficients of the delay line. The next audio sampling point of the current audio frame is taken as the current audio sampling point of the current audio frame to obtain the new parameter coefficients of the delay line. Then, the sampling data of the current audio sampling point of the current audio frame is multiplied by the first positive parameter to obtain the first product.

[0135] If the current audio sample point of the current audio frame is the last audio sample point of the current audio frame, then the output data of all audio sample points of the current audio frame will be used as the processed current audio frame.

[0136] After obtaining the adjusted first loudness gain value, separate real-time low-frequency enhancement filters can be used for the left and right channels. Compared with traditional low-frequency enhancement filters, real-time filters can solve signal abrupt changes in frame processing by buffering old values, updating parameters, updating the total gain, filtering samples one by one, updating the delay line, and smoothing frames one by one in order to ensure the consistency of the effect and the smoothness of the transition between two frames. Finally, the final stereo frame data is output.

[0137] When determining the filter coefficients based on the adjusted first loudness gain value, the filter center frequency (e.g., 250Hz), slope parameter S (e.g., 0.4), and first loudness gain value can be used as a reference. and sampling rate The forward and feedback parameters of the filter are determined using the standard second-order IIR low-shelf formula.

[0138] Initial filter forward and feedback parameters, delay line parameter coefficients, and current loudness gain. As shown below:

[0139]

[0140]

[0141]

[0142]

[0143] in, These are the filter's positive parameters, which include a first positive parameter b0bass, a second positive parameter b1bass, and a third positive parameter b2bass. These are the filter feedback parameters, which include the first feedback parameter a0bass, the second feedback parameter a1bass, and the third feedback parameter a2bass. , , , These are the four input and output values ​​required for filter calculation: xn1Bass, the sampled data from the first audio sample point before the current audio sample point in the current audio frame; xn2Bass, the sampled data from the second audio sample point before the current audio sample point in the current audio frame; yn1Bass, the output data from the first audio sample point before the current audio sample point in the current audio frame; and yn2Bass, the output data from the second audio sample point before the current audio sample point in the current audio frame. This is the gain value for the current loudness, initially set to -1 dB. The reason for not setting it to 0 is to prevent the low-volume sound from being boosted to 0 in the first frame, which would cause... , , , The issue of subsequent updates not being able to proceed normally.

[0144] Current audio frame Separate into left and right channels, denoted as and .in Represents left channel data. This represents the data for the right channel.

[0145] Left channel For example, in calculation, you can set , Calculated for the previous audio frame Compare it with the current A comparison is performed. If the two are consistent, the output data of the current audio sampling point of the current audio frame is calculated. If they are inconsistent, the output data is calculated based on the current audio frame. Update positive parameters and feedback parameters .

[0146] For sample-by-sample filtering, the standard IIR filter function can be used. From the left channel... Data is collected point by point to obtain the sampling data of the current audio sampling point of the left channel of the current audio frame. Calculate using the following formula with The output data of the current audio sample point of the corresponding current audio frame .

[0147]

[0148] The right channel uses the same method for input and output calculations, which will not be elaborated here.

[0149] Then, delay line updates are performed point-by-point.

[0150] If the current audio sample point of the current audio frame is the last audio sample point of the current audio frame, then the output data of all audio sample points of the current audio frame will be used as the processed current audio frame.

[0151] If the current audio sampling point of the current audio frame is not the last audio sampling point of the current audio frame, update the parameter coefficients of the delay line based on the output data of the current audio sampling point of the current audio frame to obtain the new parameter coefficients of the delay line. Then, take the next audio sampling point of the current audio frame as the current audio sampling point of the current audio frame, obtain the new parameter coefficients of the delay line, and then multiply the sampling data of the current audio sampling point of the current audio frame by the first positive parameter to obtain the first product.

[0152] Each audio sample point in the current audio frame is processed based on the parameter coefficients of the delay line to obtain the output data for that audio sample point. After obtaining the output data of an audio sample point, before calculating the output data of the next audio sample point, it is necessary to update the parameter coefficients of the delay line to avoid the problem of incorrect output data for the audio sample point if the parameter coefficients of the delay line are not the same as those of the delay line corresponding to the next audio sample point (e.g., the parameter coefficients of the delay line corresponding to a previous audio sample point or the initial parameter coefficients of the delay line). Therefore, the parameter coefficients of the delay line can be updated based on the output data of the current audio sample point of the current audio frame to obtain the new parameter coefficients of the delay line. The next audio sample point of the current audio frame is then used as the current audio sample point of the current audio frame to obtain the new parameter coefficients of the delay line. Finally, the sample data of the current audio sample point of the current audio frame is multiplied by the first positive parameter to obtain the first product.

[0153] In one possible implementation, the parameter coefficients of the delay line are updated based on the output data of the current audio sample point of the current audio frame to obtain the new parameter coefficients of the delay line, including:

[0154] The first parameter coefficient is used as the new second parameter coefficient, the sampling data of the current audio sampling point of the current audio frame is used as the new first parameter coefficient, the third parameter coefficient is used as the new fourth parameter, and the output data of the current audio sampling point of the current audio frame is used as the new third parameter, thus obtaining the parameter coefficients of the new delay line.

[0155] After the sample-by-sample filtering is completed, i.e., after obtaining the output data of the current audio sampling point of the current audio frame, the parameter coefficients of the delay line are updated, as shown in the following formula:

[0156]

[0157]

[0158]

[0159]

[0160] In one possible implementation, after obtaining the processed current audio frame, the following is also included:

[0161] When the first loudness gain value or the second loudness gain value is updated after adjustment, the sampled data of the last audio sample point of the current audio frame is used as the first parameter coefficient of the delay line, the sampled data of the second to last audio sample point of the current audio frame is used as the second parameter coefficient of the delay line, the output data of the last audio sample point of the current audio frame is used as the third parameter coefficient of the delay line, and the output data of the second to last audio sample point of the current audio frame is used as the fourth parameter coefficient of the delay line.

[0162] After obtaining the processed current audio frame, the next audio frame is processed. Using the above steps, adjusted first loudness gain and adjusted second loudness gain values ​​are obtained for the next audio frame. Based on the previously obtained adjusted first and second loudness gain values, it is determined whether an update has occurred. If an update has occurred, to maintain the consistency of the inter-frame reference logic and avoid errors... , , , Since the data might be buffered from the previous frame or still contain initial values, the delay line data needs to be updated. Specifically, the sampled data from the last audio sample point of the current audio frame is used as the first parameter coefficient of the delay line; the sampled data from the second-to-last audio sample point of the current audio frame is used as the second parameter coefficient; the output data from the last audio sample point of the current audio frame is used as the third parameter coefficient; and the output data from the second-to-last audio sample point of the current audio frame is used as the fourth parameter coefficient. The formula for updating the delay line parameter coefficients is as follows:

[0163]

[0164]

[0165]

[0166]

[0167] In the formula, This refers to the sampled data of the second-to-last audio sample point in the current audio frame. This refers to the sampled data of the last audio sample point in the current audio frame. This is the output data for the second-to-last audio sample point of the current audio frame. This is the output data for the last audio sample point of the current audio frame.

[0168] In one possible implementation, after obtaining the processed current audio frame, the following is also included:

[0169] If the current audio frame is the first audio frame, then N zeros are used as the output data of the first N audio sampling points of the current smoothed audio frame, and the output data of the first MN audio sampling points of the current audio frame are used as the output data of the last MN audio sampling points of the current smoothed audio frame, thus obtaining the current smoothed audio frame; where M is the total number of audio sampling points of the current audio frame after processing, N is the preset number of audio sampling points, and N < M.

[0170] If the current audio frame is not the first audio frame, the output data of the last N audio sampling points of the previous audio frame is used as the first data, and the output data of the first N audio sampling points of the current audio frame is used as the second data. The average value of the first data is determined to obtain the first average value. The average value of the second data is determined to obtain the second average value. The difference between the first average value and the second average value is determined to obtain the target difference value. The product of the output data of each audio sampling point of the first data and the exponentiation value is determined to obtain the output data of the first N audio sampling points of the current smoothed audio frame. The output data of the first MN audio sampling points of the current audio frame is used as the output data of the last MN audio sampling points of the current smoothed audio frame to obtain the current smoothed audio frame. The exponentiation value is obtained by exponentiation with the target difference value multiplied by the target ratio value to the base 10. The target ratio value is the ratio of the position of the output data of the audio sampling points of the first data in the first data to the total number of data in the first data.

[0171] Standard audio loudness limiting forcibly restricts the audio amplitude to between -1 and 1, which can cause listening discomfort. The frame-by-frame smoothing provided in this application can mitigate this discomfort. Furthermore, because the loudness boosting method and magnitude differ for each frame, noise is easily generated at the transition points between frames during streaming processing. In response to this, this application addresses loudness abrupt changes based on sampling point buffers.

[0172] Taking a current audio frame with a total number of audio sampling points M of 1024 and a preset number of audio sampling points N of 256 as an example, the smoothing process is explained. In the calculation of the first frame, since there is no reference value from the previous frame, it is necessary to delay the first frame by 256 sampling points with a value of 0, denoted as... The subsequent sampled signal is truncated and buffered for output. The player's actual first frame output consists of the first 256 zero-sampled values ​​and the output data from the first 1024-256 samples of the first frame. The last 256 samples of the first frame are retained, and their average value is calculated to obtain the first average value. Calculate the average value of the output data from the first 256 sampling points of the second frame to obtain the second average value. The sample from the previous frame gradually decreases or increases until it matches the average loudness of the second frame. The formula is:

[0173]

[0174] In the formula, n represents the number of audio frames; n=1 indicates the first audio frame. Before the first frame A sampling point where all values ​​are 0. This refers to the initial state variable name in the filter. This represents the total number of audio sample points in the audio frame, which can be 1024. This is the preset number of audio sampling points, used for the soft limit of loudness abrupt changes between two frames; it can be 256. It is a counting unit used to calculate the loudness change weight at each point, with a value ranging from 1 to... For example, if the output data of the audio sample point of the first data is the first in the first data, then For example, if the output data of the audio sample point of the first data is the last one in the first data, then... for . This represents the audio signal from the previous frame. This indicates the first sample point of the audio frame currently being smoothed. Sampling points Indicates the previous audio frame sampling point to Sampling point. This is a power operation with base 10, where the target difference is multiplied by the target ratio and raised to the power of 10. - Let k be the target difference, and k / block be the target ratio. The change in loudness can be converted into an amplitude value through exponentiation. "Splicing" means that in a single audio frame, the content consists of two parts: one part comes from the previous frame, and the other part comes from the current frame. These two parts are then spliced ​​together to form a complete audio file. , The audio frame currently being smoothed.

[0175] The following example illustrates this: the average loudness of the last 256 points in the first frame is -0.8 dB, and the average loudness of the first 256 points in the second frame is -0.2 dB, a difference of -0.6 dB. The first frame needs to reach the threshold of -0.2 dB in 256 points. Based on the -0.6 dB difference, the loudness of each of the 256 points in the first frame is gradually increased until it reaches -0.2 dB, which ensures both the loudness continuity between the two frames and that the overall dynamics remain within a certain range.

[0176] This application can adapt to the real-time frame processing of the player by streaming data. Based on frame-level analysis, it determines the gain value for low-frequency enhancement and adaptively adjusts the boost level of low frequencies. This not only ensures the dynamic range of the original audio but also enhances the atmosphere of the accompaniment, thereby stimulating users' interest in karaoke and increasing the user usage rate of the karaoke module in the player's app.

[0177] The above describes an audio processing method provided by an embodiment of this application. The following will describe a system that performs the above audio processing method.

[0178] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an audio processing system provided in an embodiment of this application. Figure 2 As shown, the audio processing system includes:

[0179] The acquisition module 201 is used to acquire the current audio frame; wherein, the current audio frame includes sampling data of multiple audio sampling points.

[0180] The first determining module 202 is used to determine the loudness of the current audio frame and the average peak loudness of the current frame; wherein, the average peak loudness of the current frame is the average of the peak loudness of the sampled data of all audio sampling points of the current audio frame.

[0181] The second determining module 203 is used to determine the difference between the first target loudness and the loudness of the current audio frame to obtain a first difference, determine the difference between the second target loudness and the average peak loudness of the current frame to obtain a second difference, and take the smaller value of the first difference and the second difference as the target loudness gain value.

[0182] The output module 204 is used to perform audio enhancement processing on the current audio frame based on the target loudness gain value to obtain the processed current audio frame.

[0183] In one possible implementation, the audio processing system provided in this application further includes:

[0184] The proportion calculation module is used to filter the current audio frame after taking the smaller of the first difference and the second difference as the target loudness gain value to obtain the target frequency signal; wherein, the target frequency signal is an audio signal with a frequency lower than a preset frequency; determine the proportion of the target frequency signal in the current audio frame; if the proportion is less than the proportion threshold, determine the audio enhancement ratio coefficient based on the proportion, and take the product of the audio enhancement ratio coefficient and the target loudness gain value as the first loudness gain value; if the proportion is not less than the proportion threshold, take the proportion as the weight of the second difference, take the difference between 1 and the proportion as the weight of the first difference, add the product of the first difference and the weight of the first difference to the product of the second difference and the weight of the second difference to obtain a summation value, and take half of the summation value as the second loudness gain value.

[0185] Output module 204 is specifically used for:

[0186] If the proportion is less than the proportion threshold, the current audio frame is enhanced based on the first loudness gain value to obtain the processed current audio frame.

[0187] If the proportion is not less than the proportion threshold, the current audio frame is enhanced based on the second loudness gain value to obtain the processed current audio frame.

[0188] In one possible implementation, the percentage calculation module is specifically used for:

[0189] If the proportion is less than the target value, then the audio enhancement ratio coefficient is set to 1;

[0190] If the proportion is not less than the target value and the proportion is less than the proportion threshold, then the audio enhancement ratio coefficient is determined to be a value greater than 0 and less than 1.

[0191] If the proportion is not less than the proportion threshold, then the audio enhancement ratio coefficient is determined to be 0.

[0192] In one possible implementation, the audio processing system provided in this application further includes:

[0193] The gain adjustment module is used to determine the average loudness and average peak loudness of audio frames in a sliding window after obtaining a first loudness gain value or a second loudness gain value. The audio frames in the sliding window include the current audio frame and p audio frames preceding it. The average loudness of the audio frames is the average loudness of the audio frames in the sliding window, and the average peak loudness is the average peak loudness of the audio frames in the sliding window. The module determines the difference between the average peak loudness of the current frame and the average peak loudness, obtaining a third difference. It also determines the difference between the average loudness of the audio frames and the average peak loudness, obtaining a fourth difference. If the absolute value of both the third and fourth differences is greater than a sudden change threshold, the sum of the first loudness gain value and the gain adjustment value is used as the adjusted first loudness gain value, and the sum of the second loudness gain value and the gain adjustment value is used as the adjusted second loudness gain value. The gain adjustment value is positive when both the third and fourth differences are positive, and negative when both are negative.

[0194] Optionally, output module 204 is also used for:

[0195] When the proportion is less than the proportion threshold, the current audio frame is enhanced based on the adjusted first loudness gain value to obtain the processed current audio frame.

[0196] If the proportion is not less than the proportion threshold, the current audio frame is enhanced based on the second loudness gain value to obtain the processed current audio frame, including:

[0197] If the proportion is not less than the proportion threshold, the current audio frame is enhanced based on the adjusted second loudness gain value to obtain the processed current audio frame.

[0198] Optionally, output module 204 is also used for:

[0199] Using 10 as the base and the ratio of the adjusted second loudness gain value to 20 as the exponent, perform a power operation to obtain the loudness after linear gain conversion;

[0200] Multiply the loudness after linear gain conversion with the current audio frame to obtain the processed current audio frame.

[0201] Optionally, output module 204 is also used for:

[0202] Obtain the parameter coefficients of the delay line; where the parameter coefficients of the delay line include the sampling data and output data of historical audio sampling points;

[0203] The filter coefficients are determined based on the adjusted first loudness gain value;

[0204] Based on the parameter coefficients of the delay line, the filter coefficients, and the sampling data of the current audio sampling point of the current audio frame, the output data of the current audio sampling point of the current audio frame is determined.

[0205] Determine whether the current audio sample point of the current audio frame is the last audio sample point of the current audio frame; if the current audio sample point of the current audio frame is not the last audio sample point of the current audio frame, update the parameter coefficients of the delay line based on the output data of the current audio sample point of the current audio frame to obtain the new parameter coefficients of the delay line, and take the next audio sample point of the current audio frame as the current audio sample point of the current audio frame. Based on the new parameter coefficients of the delay line, the filter coefficients and the sampling data of the current audio sample point of the current audio frame, determine the output data of the current audio sample point of the current audio frame, and then execute the determination whether the current audio sample point of the current audio frame is the last audio sample point of the current audio frame.

[0206] If the current audio sample point of the current audio frame is the last audio sample point of the current audio frame, then the output data of all audio sample points of the current audio frame will be used as the processed current audio frame.

[0207] Optionally, output module 204 is also used for:

[0208] The first parameter coefficient is used as the new second parameter coefficient, the sampling data of the current audio sampling point of the current audio frame is used as the new first parameter coefficient, the third parameter coefficient is used as the new fourth parameter, and the output data of the current audio sampling point of the current audio frame is used as the new third parameter, thus obtaining the parameter coefficients of the new delay line.

[0209] In one possible implementation, the audio processing system provided in this application further includes:

[0210] The delay line parameter coefficient update module is used, after obtaining the processed current audio frame and updating the adjusted first loudness gain value or the adjusted second loudness gain value, to take the sampled data of the last audio sample point of the current audio frame as the first parameter coefficient of the delay line, the sampled data of the second to last audio sample point of the current audio frame as the second parameter coefficient of the delay line, the output data of the last audio sample point of the current audio frame as the third parameter coefficient of the delay line, and the output data of the second to last audio sample point of the current audio frame as the fourth parameter coefficient of the delay line.

[0211] In one possible implementation, the audio processing system provided in this application further includes:

[0212] The smoothing module, after obtaining the processed current audio frame, if the current audio frame is the first audio frame, uses N zeros as the output data of the first N audio sampling points of the current smoothed audio frame, and uses the output data of the first MN audio sampling points of the current audio frame as the output data of the last MN audio sampling points of the current smoothed audio frame, thus obtaining the current smoothed audio frame; where M is the total number of audio sampling points of the processed current audio frame, N is the preset number of audio sampling points, and N < M;

[0213] If the current audio frame is not the first audio frame, the output data of the last N audio sampling points of the previous audio frame is used as the first data, and the output data of the first N audio sampling points of the current audio frame is used as the second data. The average value of the first data is determined to obtain the first average value. The average value of the second data is determined to obtain the second average value. The difference between the first average value and the second average value is determined to obtain the target difference value. The product of the output data of each audio sampling point of the first data and the exponentiation value is determined to obtain the output data of the first N audio sampling points of the current smoothed audio frame. The output data of the first MN audio sampling points of the current audio frame is used as the output data of the last MN audio sampling points of the current smoothed audio frame to obtain the current smoothed audio frame. The exponentiation value is obtained by exponentiation with the target difference value multiplied by the target ratio value to the base 10. The target ratio value is the ratio of the position of the output data of the audio sampling points of the first data in the first data to the total number of data in the first data.

[0214] This application also provides an electronic device in its embodiments. (See reference...) Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0215] like Figure 3 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. When the electronic device is powered on, the RAM 303 also stores various programs and data required for the operation of the electronic device. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0216] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, memory cards, hard drives, etc.; and communication devices 309. Communication device 309 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0217] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the audio processing methods provided in this application.

[0218] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the audio processing methods provided in this application.

[0219] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0220] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0221] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0222] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. An audio processing method, characterized in that, include: Obtain the current audio frame; wherein, the current audio frame includes sampling data of multiple audio sampling points; Determine the loudness of the current audio frame and the average peak loudness of the current frame; wherein, the average peak loudness of the current frame is the average of the peak loudness of the sampled data of all audio sampling points of the current audio frame; The difference between the first target loudness and the loudness of the current audio frame is determined to obtain the first difference value. The difference between the second target loudness and the average peak loudness of the current frame is determined to obtain the second difference value. The smaller value between the first difference value and the second difference value is taken as the target loudness gain value. Based on the target loudness gain value, the current audio frame is subjected to audio enhancement processing to obtain the processed current audio frame.

2. The audio processing method according to claim 1, characterized in that, After taking the smaller of the first difference and the second difference as the target loudness gain value, the method further includes: The current audio frame is filtered to obtain a target frequency signal; wherein the target frequency signal is an audio signal with a frequency lower than a preset frequency. Determine the proportion of the target frequency signal in the current audio frame; When the proportion is less than the proportion threshold, an audio enhancement ratio coefficient is determined based on the proportion, and the product of the audio enhancement ratio coefficient and the target loudness gain value is used as the first loudness gain value. When the proportion is not less than the proportion threshold, the proportion is used as the weight of the second difference, the difference between 1 and the proportion is used as the weight of the first difference, the product of the first difference and the weight of the first difference is added to the product of the second difference and the weight of the second difference to obtain a summation value, and half of the summation value is used as the second loudness gain value. The step of performing audio enhancement processing on the current audio frame based on the target loudness gain value to obtain the processed current audio frame includes: When the proportion is less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the first loudness gain value to obtain the processed current audio frame. If the proportion is not less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the second loudness gain value to obtain the processed current audio frame.

3. The audio processing method according to claim 2, characterized in that, The process of determining the audio enhancement ratio coefficient based on the ratio includes: If the percentage is less than the target value, then the audio enhancement ratio coefficient is determined to be 1; If the percentage is not less than the target value and the percentage is less than the percentage threshold, then the audio enhancement ratio coefficient is determined to be a value greater than 0 and less than 1. If the percentage is not less than the percentage threshold, then the audio enhancement ratio coefficient is determined to be 0.

4. The audio processing method according to claim 2, characterized in that, After obtaining the first loudness gain value or the second loudness gain value, the method further includes: Determine the average loudness and average peak loudness of audio frames in a sliding window; wherein, the audio frames in the sliding window include the current audio frame and p audio frames preceding the current audio frame, the average loudness of audio frames is the average loudness of audio frames in the sliding window, and the average peak loudness is the average peak loudness of audio frames in the sliding window; The difference between the average peak loudness of the current frame and the average peak loudness is determined to obtain a third difference value; the difference between the average loudness of the audio frame and the average peak loudness is determined to obtain a fourth difference value. If the absolute value of the third difference is greater than the mutation threshold and the absolute value of the fourth difference is greater than the mutation threshold, the sum of the first loudness gain value and the gain adjustment value is taken as the adjusted first loudness gain value, and the sum of the second loudness gain value and the gain adjustment value is taken as the adjusted second loudness gain value; wherein, when both the third difference and the fourth difference are positive, the gain adjustment value is positive, and when both the third difference and the fourth difference are negative, the gain adjustment value is negative. When the proportion is less than the proportion threshold, the audio enhancement processing of the current audio frame is performed based on the first loudness gain value to obtain the processed current audio frame, including: When the proportion is less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the adjusted first loudness gain value to obtain the processed current audio frame. When the proportion is not less than the proportion threshold, the audio enhancement processing of the current audio frame is performed based on the second loudness gain value to obtain the processed current audio frame, including: If the proportion is not less than the proportion threshold, the current audio frame is subjected to audio enhancement processing based on the adjusted second loudness gain value to obtain the processed current audio frame.

5. The audio processing method according to claim 4, characterized in that, The step of performing audio enhancement processing on the current audio frame based on the adjusted first loudness gain value to obtain the processed current audio frame includes: Obtain the parameter coefficients of the delay line; wherein, the parameter coefficients of the delay line include the sampling data and output data of historical audio sampling points; The filter coefficients are determined based on the adjusted first loudness gain value; Based on the parameter coefficients of the delay line, the filter coefficients, and the sampling data of the current audio sampling point of the current audio frame, the output data of the current audio sampling point of the current audio frame is determined; Determine whether the current audio sampling point of the current audio frame is the last audio sampling point of the current audio frame; If the current audio sampling point of the current audio frame is not the last audio sampling point of the current audio frame, then the parameter coefficients of the delay line are updated based on the output data of the current audio sampling point of the current audio frame to obtain new parameter coefficients of the delay line. The next audio sampling point of the current audio frame is taken as the current audio sampling point of the current audio frame. Based on the new parameter coefficients of the delay line, the filter coefficients, and the sampling data of the current audio sampling point of the current audio frame, the output data of the current audio sampling point of the current audio frame is determined. Then, the step of determining whether the current audio sampling point of the current audio frame is the last audio sampling point of the current audio frame is executed. If the current audio sampling point of the current audio frame is the last audio sampling point of the current audio frame, then the output data of all audio sampling points of the current audio frame is used as the processed current audio frame.

6. The audio processing method according to claim 5, characterized in that, Following the processed current audio frame, the following is also included: When the adjusted first loudness gain value or the adjusted second loudness gain value is updated, the sampled data of the last audio sample point of the current audio frame is used as the first parameter coefficient of the delay line, the sampled data of the second-to-last audio sample point of the current audio frame is used as the second parameter coefficient of the delay line, the output data of the last audio sample point of the current audio frame is used as the third parameter coefficient of the delay line, and the output data of the second-to-last audio sample point of the current audio frame is used as the fourth parameter coefficient of the delay line.

7. The audio processing method according to any one of claims 1 to 6, characterized in that, Following the processed current audio frame, the following is also included: If the current audio frame is the first audio frame, then N zeros are used as the output data of the first N audio sampling points of the current smoothed audio frame, and the output data of the first MN audio sampling points of the current audio frame are used as the output data of the last MN audio sampling points of the current smoothed audio frame, thus obtaining the current smoothed audio frame; where M is the total number of audio sampling points of the processed current audio frame, N is the preset number of audio sampling points, and N < M. If the current audio frame is not the first audio frame, the output data of the last N audio sampling points of the previous audio frame is used as the first data, and the output data of the first N audio sampling points of the current audio frame is used as the second data. The average value of the first data is determined to obtain the first average value. The average value of the second data is determined to obtain the second average value. The difference between the first average value and the second average value is determined to obtain the target difference value. The product of the output data of each audio sampling point of the first data and the power operation value is determined to obtain the output data of the first N audio sampling points of the current smoothed audio frame. The output data of the first MN audio sampling points of the current audio frame is used as the output data of the last MN audio sampling points of the current smoothed audio frame to obtain the current smoothed audio frame. The power operation value is obtained by exponentiation with the target difference value multiplied by the target ratio value to the base 10. The target ratio value is the ratio of the position of the output data of the audio sampling points of the first data in the first data to the total number of data in the first data.

8. An audio processing system, characterized in that, include: An acquisition module is used to acquire the current audio frame; wherein, the current audio frame includes sampling data of multiple audio sampling points; The first determining module is used to determine the loudness of the current audio frame and the average peak loudness of the current frame; wherein the average peak loudness of the current frame is the average of the peak loudness of the sampled data of all audio sampling points of the current audio frame; The second determining module is used to determine the difference between the first target loudness and the loudness of the current audio frame to obtain a first difference, determine the difference between the second target loudness and the average peak loudness of the current frame to obtain a second difference, and take the smaller value between the first difference and the second difference as the target loudness gain value. The output module is used to perform audio enhancement processing on the current audio frame based on the target loudness gain value to obtain the processed current audio frame.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the audio processing method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the audio processing method as described in any one of claims 1 to 7.

Images