Audio processing method, device, equipment and medium

Abstract

The embodiment of the present application provides an audio processing method, device, equipment and medium, including: pre-processing the current audio frame according to the volume and threshold of the audio limiter to obtain at least one audio unit to be limited; processing at least one audio unit in turn, and determining the reference scaling factor of the current audio unit according to the volume peak of the current audio unit, the threshold and the buffer amplitude; performing continuity processing on the reference scaling factor of the current audio unit according to the reference scaling factor of the forward audio unit, so that the scaling factor of the sampling point at the head of the current audio unit is continuous with the reference scaling factor of the sampling point at the tail of the forward audio unit; and performing limit processing on each sampling point of the current audio unit according to the scaling factor of each sampling point of the current audio unit. The method only limits the current audio unit according to the reference scaling factor of the forward audio unit, without the information of the backward audio unit, thereby achieving zero delay in the limit processing.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio processing, and in particular, to an audio processing method, apparatus, device and medium. Background Art

[0002] Audio limiter is a software function widely used in audio production. It is mainly used to control the dynamic range of audio signals to ensure the stability and high quality of audio signals. For example, in audio and video editing scenarios, users can use audio and video editing software to process audio signals such as background music (BGM), original sound, and recordings. During the processing, the audio limiter can be used to adjust the volume to avoid popping sounds.

[0003] In the prior art, the audio limiter needs to perform bidirectional analysis on the forward and backward frames of the audio frame, determine the envelope range of the audio signal, compare the size relationship between the envelope range and the effective range, and compress the audio signal exceeding the effective range to different degrees according to the envelope size to ensure that the audio will not have popping sounds. In this method, since it is necessary to wait for the audio data of the subsequent frame to complete the accurate processing of the current audio frame, a delay will be introduced. Summary of the invention

[0004] The embodiments of the present application provide an audio processing method, apparatus, device and medium, which limit the current audio unit only according to the reference scaling factor of the forward audio unit, without the need for information of the backward audio unit, thereby achieving zero delay in the limiting processing.

[0005] In a first aspect, an embodiment of the present application provides an audio processing method, the method comprising: pre-processing a current audio frame according to a volume and a threshold of an audio limiter to obtain at least one audio unit to be limited; processing the at least one audio unit in sequence, and determining a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold of the audio limiter, and a buffer amplitude, wherein the reference scaling factor is less than or equal to 1; performing continuity processing on the reference scaling factor of the current audio unit according to a reference scaling factor of a forward audio unit adjacent to the current audio unit, to obtain a scaling factor of each sampling point of the current audio unit, wherein the scaling factor of the sampling point at the head of the current audio unit is continuous with the scaling factor of the sampling point at the tail of the forward audio unit; and performing limiting processing on each sampling point of the current audio unit according to the scaling factor of each sampling point of the current audio unit.

[0006] In some exemplary embodiments, determining the reference scaling factor of the current audio unit based on the volume peak value of the current audio unit, the threshold value of the audio limiter, and the buffer amplitude includes: calculating the sum of the volume peak value of the current audio unit and the buffer amplitude to obtain a first amplitude; when the first amplitude is greater than or equal to the threshold value of the audio limiter, calculating the ratio of the threshold value of the audio limiter to the first amplitude, and using the ratio as the reference scaling factor of the current audio unit; when the first amplitude is less than the threshold value of the audio limiter, determining that the reference scaling factor of the current audio unit is 1.

[0007] In some exemplary embodiments, determining the reference scaling factor of the current audio unit based on the volume peak value of the current audio unit, the threshold value of the audio limiter, and the buffer amplitude includes: calculating the product of the volume peak value of the current audio unit and the adjustment factor, and calculating the sum of the product and the buffer amplitude to obtain a second amplitude; when the second amplitude is greater than or equal to the threshold value of the audio limiter, calculating the ratio of the threshold value of the audio limiter to the second amplitude, and using the ratio as the reference scaling factor of the current audio unit; when the second amplitude is less than the threshold value of the audio limiter, determining that the reference scaling factor of the current audio unit is 1.

[0008] In some exemplary embodiments, determining the reference scaling factor of the current audio unit according to the volume peak value of the current audio unit, the threshold value of the audio limiter, and the buffer amplitude includes: obtaining a transformation factor through exponential operation according to the volume peak value of the current audio unit and the buffer amplitude, and the transformation factor is greater than 0; calculating the sum of the volume peak value of the current audio unit and a third amplitude to obtain a fourth amplitude, and the third amplitude is the product of the volume peak value of the current audio unit and the transformation factor; when the fourth amplitude is greater than or equal to the threshold value of the audio limiter, calculating the ratio of the threshold value of the audio limiter to the fourth amplitude, and using the ratio as the reference scaling factor of the current audio unit; when the fourth amplitude is less than the threshold value of the audio limiter, determining that the reference scaling factor of the current audio unit is 1.

[0009] In some exemplary embodiments, the base scaling factor of the current audio unit is continuously processed according to the base scaling factor of the forward audio unit adjacent to the current audio unit to obtain the scaling factor of each sampling point of the current audio unit, including: determining the position of the buffer area of ​​the current audio unit, the starting position of the buffer area of ​​the current audio unit being the starting position of the current audio unit; interpolating the scaling factors of the sampling points in the buffer area according to the base scaling factor of the current audio unit and the base scaling factor of the forward audio unit, and determining the scaling factors of the sampling points in the buffer area according to the interpolation results.

[0010] In some exemplary embodiments, determining the scaling factor of the sampling point within the buffer area based on the interpolation result includes: for each sampling point within the buffer area, calculating the ratio of the threshold of the audio limiter to the amplitude of the sampling point, the ratio being the maximum scaling factor of the sampling point; when the interpolation scaling factor of the sampling point is less than or equal to the maximum scaling factor of the sampling point, determining the interpolation scaling factor of the sampling point to be the scaling factor of the sampling point; when the interpolation scaling factor of the sampling point is greater than the maximum scaling factor of the sampling point, determining the maximum scaling factor of the sampling point to be the scaling factor of the sampling point.

[0011] In some exemplary embodiments, after determining the scaling factors of the sampling points in the buffer area according to the interpolation results, the method further includes: smoothing the scaling factors of the sampling points in the buffer area.

[0012] In some exemplary embodiments, determining the position of the buffer area of ​​the current audio unit includes: determining the length of the buffer area of ​​the current audio unit based on a first preset length of the buffer area and / or the position of the volume peak of the current audio unit, wherein the end position of the buffer area cannot exceed the position of the volume peak of the current audio unit.

[0013] In some exemplary embodiments, determining the length of the buffer area of ​​the current audio unit based on a first preset length of the buffer area and / or the position of the volume peak of the current audio unit includes taking the minimum value of the first preset length and the position of the volume peak of the current audio unit as the length of the buffer area of ​​the current audio unit.

[0014] In some exemplary embodiments, the preprocessing of the current audio frame according to the volume and threshold of the audio limiter to obtain at least one audio unit to be limited includes: extracting original audio data from the current audio frame, scaling the original audio data according to the volume of the audio limiter to obtain audio data to be limited; when the length of the current audio frame is less than or equal to a second preset length, determining the audio data to be limited as an audio unit; when the length of the current audio frame is greater than the second preset length, dividing the audio data to be limited into multiple audio units according to the second preset length.

[0015] In some exemplary embodiments, after limiting each sampling point of the current audio unit according to the scaling factor of each sampling point of the current audio unit, the method further includes: determining that the current audio unit is the last audio unit in the current audio frame; splicing the limiting results of all audio units in the current audio frame to obtain the limiting result of the current audio frame; and outputting the limiting result of the current audio frame.

[0016] In some exemplary embodiments, when the current audio unit is the first audio unit, before performing continuity processing on the baseline scaling factor of the current audio unit based on the baseline scaling factor of the forward audio unit adjacent to the current audio unit, it also includes: determining the baseline scaling factor of the forward audio unit of the first audio unit based on the baseline scaling factor of the first audio unit.

[0017] In some exemplary embodiments, the buffer amplitude is associated with a threshold value of the audio limiter.

[0018] In a second aspect, an embodiment of the present application provides an audio processing device, the device comprising: a preprocessing module, a determination module, a continuity processing module and a limit module. The preprocessing module is used to preprocess the current audio frame according to the volume and threshold of the audio limiter to obtain at least one audio unit to be limited; the determination module is used to process the at least one audio unit in sequence, and determine the reference scaling factor of the current audio unit according to the volume peak value of the current audio unit, the threshold of the audio limiter and the buffer amplitude, and the reference scaling factor is less than or equal to 1; the continuity processing module is used to perform continuity processing on the reference scaling factor of the current audio unit according to the reference scaling factor of the forward audio unit adjacent to the current audio unit, and obtain the scaling factor of each sampling point of the current audio unit, and the scaling factor of the head sampling point of the current audio unit is continuous with the scaling factor of the sampling point at the end of the forward audio unit; the limit module is used to perform limit processing on each sampling point of the current audio unit according to the scaling factor of each sampling point of the current audio unit.

[0019] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method described in the first aspect or each implementation method above.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the method described in the first aspect or each implementation method above.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method of the first aspect or each implementation method described above.

[0022] Through the technical solution provided by the embodiment of the present application, according to the volume and threshold of the audio limiter, the current audio frame is preprocessed to obtain at least one audio unit to be limited; at least one audio unit is processed in sequence, and the reference scaling factor of the current audio unit is determined according to the volume peak value of the current audio unit, the threshold and the buffer amplitude; according to the reference scaling factor of the adjacent forward audio unit before the current audio unit, the reference scaling factor of the current audio unit is processed for continuity to obtain the scaling factor of each sampling point of the current audio unit, and the scaling factor of the sampling point at the head of the current audio unit is continuous with the reference scaling factor of the sampling point at the tail of the forward audio unit; according to the scaling factor of each sampling point of the current audio unit, each sampling point of the current audio unit is limited. This method only limits the current audio unit according to the reference scaling factor of the forward audio unit, without the information of the backward audio unit, thereby achieving zero delay in the limiting process. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flowchart of the audio processing method provided in Embodiment 1 of the present application;

[0025] Figure 2 This is a schematic diagram of the interpolation results of the buffer area;

[0026] Figure 3 A flowchart of the audio processing method provided in Embodiment 2 of the present application;

[0027] Figure 4 A schematic diagram of the structure of an audio processing device provided in Embodiment 3 of the present application;

[0028] Figure 5 A structural schematic diagram of an electronic device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0031] In order to facilitate the understanding of the embodiments of the present application, before describing the various embodiments of the present application, some concepts involved in all the embodiments of the present application are first appropriately explained.

[0032] Gain: The amount by which an audio signal is amplified or attenuated, measured in decibels (dB).

[0033] Popping: A distortion phenomenon that occurs when an audio signal exceeds its dynamic range (for example, [-1,1]), which sounds like a popping or electric current.

[0034] Zero latency: There is no time delay between the processed audio signal and the input signal. When the audio signal is processed in frames, zero latency can mean that there is no delay between the processed audio frame and the input audio frame.

[0035] Sampling point: In digital audio, each sampling point represents the amplitude of the audio signal at a specific point in time or at a specific time. It is the basic unit for converting continuous analog signals into discrete digital signals.

[0036] Audio frame: A fixed-length segment of audio data, typically used in digital audio processing.

[0037] Sub-audio frame: A subset of an audio frame that contains smaller blocks of audio data than an audio frame.

[0038] Previous frame: The audio frame that is adjacent to the current audio frame (located before the current audio frame in time).

[0039] Backward frame: The audio frame that is adjacent to the current audio frame (located after the current audio frame in time).

[0040] Envelope: The outline of an audio signal, used to describe the dynamic changes of an audio signal.

[0041] Scaling factor: A factor used to adjust the amplitude of an audio signal.

[0042] Look-ahead: An audio processing technique that optimizes current processing by analyzing future audio signals in advance.

[0043] An embodiment of the present application provides an audio processing method, which can be applied in an audio editing scenario or an audio and video editing scenario.

[0044] Audio editing refers to the use of audio editing software to perform various processing operations on audio files, including but not limited to recording, editing, merging, mixing, noise reduction, adding sound effects and background music, adjusting volume and playback speed, etc. These operations are aimed at improving audio quality and meeting specific application requirements, such as music production, dubbing, advertising audio production, etc.

[0045] Audio and video editing refers to the process of using audio and video editing software or tools to cut, merge, reorganize, and adjust collected or shot video and audio materials to generate new, coherent and artistic audio and video works.

[0046] The materials for audio and video editing include original video files, original audio files, and pictures. The audio and video editing process includes audio editing and audio mixing. Audio editing includes editing, noise reduction, volume adjustment, and other processing of audio files to ensure the clarity of the audio and synchronization with the video.

[0047] Audio and video editing is widely used in the production of short videos and social media content, music production and mixing, course recording and editing, etc.

[0048] Through the above description, in audio editing scenarios or audio and video editing scenarios, the audio volume needs to be adjusted. Too high a volume may cause the audience to feel harsh or uncomfortable, while too low a volume may make the content difficult to hear. Audio volume adjustment can also be described as audio gain adjustment. Both volume and gain can describe the sound volume of the audio. Volume is a linear unit and gain is a logarithmic unit. The two can be converted to each other. For the convenience of understanding, the volume in linear units is mainly explained in the embodiments of this application.

[0049] To support high volume (gain) adjustment while avoiding popping sounds, existing technologies require processing through compression algorithms such as audio limiters. The audio limiter monitors the input audio signal and starts working when the signal amplitude reaches the set threshold. It quickly reduces the signal gain to keep the signal amplitude below the threshold.

[0050] The audio limiter includes the following two working parameters: volume and threshold. The volume represents the multiple of the audio waveform scaling, and the threshold represents the output range of the audio waveform. The threshold is a key parameter of the audio limiter and needs to be adjusted according to the actual situation. A too low threshold may cause the audio signal to be over-compressed and lose its dynamics, while a too high threshold may not effectively prevent popping and distortion.

[0051] Existing audio limiters are divided into two types: real-time and offline. Real-time audio limiters are suitable for editing scenes, but they will introduce a slight delay because they need to perform two-way analysis on the audio frame. Among them, two-way analysis refers to analyzing the forward audio frame and the backward audio frame of the current audio frame, obtaining the envelope range through two-way analysis, comparing the size relationship between the envelope range and the effective range, and compressing the audio signal that exceeds the effective range to different degrees according to the envelope size to ensure that the audio will not have popping sounds.

[0052] Real-time audio limiters usually use look-ahead technology to optimize the processing of the current frame by pre-analyzing the upcoming audio frames. This two-way analysis method allows the audio limiter to predict future volume peaks and adjust the gain in advance when necessary, effectively preventing audio clipping and distortion. Since it is necessary to wait for the data of the subsequent frames to complete the accurate processing of the current frame, a one-frame delay is introduced.

[0053] An embodiment of the present application provides an audio processing method, which performs audio limiting processing based on a forward frame, thereby achieving zero delay, that is, there is no delay between the output frame and the input frame.

[0054] The method of the embodiment of the present application can be applied to audio editing software or audio and video editing software, and the audio and video editing software and the audio editing software can be applications running in a terminal device, or can also be implemented as a small program or application module in a host application. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, a gaming device, a smart home, a vehicle terminal, an aircraft, etc.

[0055] Figure 1 This is a flowchart of the audio processing method provided in Example 1 of the present application. This embodiment is described by taking audio and video editing software as an example. Figure 1 As shown, the method provided in this embodiment includes the following steps.

[0056] S101. Preprocess a current audio frame according to a volume and a threshold of an audio limiter to obtain at least one audio unit to be limited.

[0057] The user imports the audio file to be processed into the audio and video editing software. The audio file may be a file downloaded by the user from the audio and video editing software, a file downloaded from other platforms, or a file recorded and uploaded by the user himself.

[0058] The volume and threshold of the audio limiter can be set by the user or use the default value. Alternatively, the volume of the audio limiter is set by the user and the threshold of the audio limiter uses the default value.

[0059] An audio file contains at least one audio frame. In actual scenarios, the length (or size) of audio frames varies. Directly processing audio frames may result in inconsistent limiting effects for audio frames of different lengths, and the audio quality cannot be guaranteed. The length of an audio frame may be related to the audio encoding format, sampling rate, number of channels, bit rate, and bit rate.

[0060] In order to adapt to audio frames of different lengths and ensure the quality of audio frames of various lengths, in this embodiment, the audio frame can be divided into multiple audio units. The audio unit is the basic processing unit of the audio processing method provided in the embodiment of the present application. The audio unit can also be called a sub-audio frame.

[0061] The audio and video editing software processes each audio frame in the audio file in turn, and calls the currently processed audio frame the current audio frame. The audio and video editing software preprocesses the current audio frame to obtain at least one audio unit to be limited.

[0062] In one implementation, original audio data is extracted from the current audio frame, and the original audio data is scaled according to the volume of the audio limiter to obtain audio data to be limited; when the length of the current audio frame is less than or equal to a second preset length, the audio data to be limited is determined as an audio unit; when the length of the current audio frame is greater than the second preset length, the audio data to be limited is divided into multiple audio units according to the second preset length.

[0063] For example, if the volume of the audio limiter is set to 200% and the threshold is set to 1, for the sampling point with a peak value of 0.6 in the original audio data, after scaling by 200%, the peak value of the sampling point becomes 1.2 (i.e. the volume is doubled), and the scaled volume exceeds the threshold range. Since the amplitude (or amplitude) of the adjusted audio data may exceed the set threshold, subsequent limit processing is required to smoothly limit the exceeding part within the threshold range.

[0064] The second preset length is the length of the audio unit to be limited. The second preset length can be adjusted according to the application scenario. The second preset length can achieve the best effect after limiting. Exemplarily, the second preset length is 10 milliseconds (ms).

[0065] For example, the length of an audio frame is 50 milliseconds and the second preset length is 10 milliseconds, then the audio frame is divided into five audio units of 10 milliseconds.

[0066] The audio and video editing software compares the length of the current audio frame with the second preset length. When the length of the current audio frame is less than or equal to the second preset length, the current audio frame does not need to be segmented, and the audio data to be limited is directly determined as one audio unit. When the length of the current audio frame is greater than the second preset length, the audio data to be limited is divided into multiple audio units according to the second preset length.

[0067] S102: Process the at least one audio unit in sequence, and determine a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold of an audio limiter, and a buffer amplitude, where the reference scaling factor is less than or equal to 1.

[0068] In the limiting processing stage, the at least one audio unit will be processed in a loop. The essence of the limiting processing is to use a suitable scaling factor to limit each sampling point in the audio unit to ensure that the audio unit is within the threshold range of the audio limiter, while ensuring the continuity between the audio unit and the forward audio unit and the relative relationship between the sampling points in the audio unit.

[0069] The concept of a forward audio unit is similar to that of a forward frame, and refers to an adjacent audio unit before a current audio unit. For example, audio frame N is divided into four audio units in order, namely: audio unit 1, audio unit 2, audio unit 3, and audio unit 4. Then, audio unit 1 is the forward audio unit of audio unit 2, audio unit 2 is the forward audio unit of audio unit 3, and the forward audio unit of audio unit 1 is the last audio unit of audio frame N-1.

[0070] The continuity between the audio unit and the forward audio unit refers to the continuity of the head of the audio unit and the tail of the forward audio unit, that is, the amplitudes of the sampling points at the head of the audio unit and the tail of the forward audio unit are continuous or non-jumpy. The continuity ensures that the audio signal will not be broken or jumped during playback, thereby maintaining stable and clear sound quality.

[0071] It is understandable that the original audio signals corresponding to the audio unit and the forward audio unit are continuous. After the limiting processing, the audio signals after the audio unit and the forward audio unit are limited should be kept as continuous as possible, so as to ensure the stability and clarity of the sound quality.

[0072] An audio unit includes multiple sampling points, and the number of sampling points is related to the sampling rate. In audio processing, the sampling rate is a key parameter that determines the number of samples (also called sampling points) extracted from the original analog audio signal per second.

[0073] For example, common sampling rates of audio frames are 16kHz (Hertz) and 24kHz. A 16kHz sampling rate means that 16,000 sample points are extracted from the audio signal per second, and 160 sample points are included in a 10 millisecond audio unit. A 24kHz sampling rate means that 22,000 sample points are extracted from the audio signal per second, and 240 sample points are included in a 10 millisecond audio unit.

[0074] The relative relationship of the sampling points in the audio unit refers to the relative size relationship of the amplitudes of all the sampling points in the audio unit. It can be understood that the amplitudes of all the sampling points in the original audio signal corresponding to the audio unit have a relative size relationship. After the limiting processing, the relative size relationship of the amplitudes of all the sampling points in the audio unit should be kept as consistent as possible with the relative size relationship of the amplitudes of all the sampling points in the original audio signal to avoid distortion of the audio signal after the limiting processing.

[0075] In this embodiment, a reference scaling factor is defined for each audio unit, and the reference scaling factor of each audio unit is calculated independently. Therefore, the reference scaling factors corresponding to different audio units may be different or the same, and the reference scaling factor of the audio unit is less than or equal to 1. All sampling points in each audio unit use the same reference scaling factor, so the reference scaling factor of the audio unit can also be described as the reference scaling factor of the sampling point of the audio unit.

[0076] The audio and video editing software compares the amplitudes of all sampling points in the current audio unit and takes the maximum amplitude as the volume peak of the current audio unit.

[0077] The simplest calculation method of the reference scaling factor of the current audio unit is: when the volume peak of the current audio unit is greater than or equal to the threshold of the audio limiter, the ratio of the volume peak of the audio limiter to the current audio unit is calculated, and the ratio is used as the reference scaling factor. If the volume peak of the current audio unit is less than the threshold of the audio limiter, the reference scaling factor of the current audio unit is determined to be 1. However, this calculation method is difficult to cope with the situation where the waveform peak of the subsequent audio signal continues to increase, which will cause the amplitudes of a large number of sampling points after limiting to be concentrated near the threshold, and the relative size relationship between the amplitudes of the sampling points is lost, and popping may occur.

[0078] In order to solve this problem, a buffer amplitude is introduced in this embodiment. The buffer amplitude is a fixed amplitude, which is used to increase the peak volume of the current audio unit to cope with any waveform that may appear in the future. By introducing the buffer amplitude, the calculated baseline scaling factor of the current audio unit is smaller than the ratio of the threshold of the audio limiter to the peak volume of the current audio unit, that is, a greater compression is performed relative to the ratio of the threshold of the audio limiter to the peak volume of the current audio unit.

[0079] In one implementation, the buffer amplitude is associated with a threshold of the audio limiter. Exemplarily, the buffer amplitude is 5% to 20% of the threshold of the audio limiter.

[0080] There are many ways to calculate the baseline scaling factor, and the overall form can be expressed as the following formula (1):

[0081] (1)

[0082] in, represents the base scaling factor, Indicates the threshold of the audio limiter. Indicates the peak volume of the current audio unit. Indicates is a function of a variable, Indicates is a function of a variable.

[0083] The first calculation method: calculate the sum of the volume peak value and the buffer amplitude of the current audio unit to obtain a first amplitude; when the first amplitude is greater than or equal to the threshold of the audio limiter, calculate the ratio of the threshold of the audio limiter to the first amplitude, and use the ratio as the reference scaling factor of the current audio unit; when the first amplitude is less than the threshold of the audio limiter, determine that the reference scaling factor of the current audio unit is 1.

[0084] The calculation method can be expressed by the following formula (2):

[0085] (2)

[0086] in, represents the base scaling factor, Indicates the threshold of the audio limiter. Indicates the peak volume of the current audio unit. Indicates the buffer amplitude, represents the first amplitude mentioned above.

[0087] The second calculation method: calculate the product of the volume peak value of the current audio unit and the adjustment factor, calculate the sum of the product and the buffer amplitude to obtain the second amplitude; when the second amplitude is greater than or equal to the threshold of the audio limiter, calculate the ratio of the threshold of the audio limiter to the second amplitude, and use the ratio as the reference scaling factor of the current audio unit; when the second amplitude is less than the threshold of the audio limiter, determine that the reference scaling factor of the current audio unit is 1.

[0088] The calculation method can be expressed by the following formula (3):

[0089] (3)

[0090] in, represents the base scaling factor, Indicates the threshold of the audio limiter. Indicates the peak volume of the current audio unit. Indicates the buffer amplitude, represents the adjustment factor, represents the second amplitude mentioned above.

[0091] The third calculation method: according to the volume peak value and buffer amplitude of the current audio unit, a conversion factor is obtained by exponential operation, and the conversion factor is greater than 0; the sum of the volume peak value of the current audio unit and the third amplitude is calculated to obtain the fourth amplitude, and the third amplitude is the product of the volume peak value of the current audio unit and the conversion factor. When the fourth amplitude is greater than or equal to the threshold of the audio limiter, the ratio of the threshold of the audio limiter to the fourth amplitude is calculated, and the ratio is used as the reference scaling factor of the current audio unit; when the fourth amplitude is less than the threshold of the audio limiter, the reference scaling factor of the current audio unit is determined to be 1.

[0092] Exemplarily, the reference scaling factor of the current audio unit may be calculated by the following formula (4):

[0093] (4)

[0094] in, represents the base scaling factor, Indicates the threshold of the audio limiter. Indicates the peak volume of the current audio unit. Indicates the buffer amplitude, represents the adjustment factor, e represents the exponential operation, , , is the above transformation factor, It is understood that the calculation method of the transformation factor is not limited to the method shown in formula (4), which is only an example.

[0095] The above formula (3) is a linear transformation, and formula (4) is a nonlinear transformation.

[0096] In this embodiment, by introducing the buffer amplitude, the amplitude of the current audio unit can be limited within the threshold of the audio limiter without being based on subsequent audio units, and the relative size relationship between the amplitudes of the sampling points in the current audio unit can be guaranteed.

[0097] S103. Perform continuity processing on the reference scaling factor of the current audio unit according to the reference scaling factor of the forward audio unit adjacent to the current audio unit to obtain the scaling factor of each sampling point of the current audio unit, wherein the scaling factor of the sampling point at the head of the current audio unit is continuous with the scaling factor of the sampling point at the tail of the forward audio unit.

[0098] In order to ensure the continuity between audio units, the continuity of the scaling factors of the sampling points between audio units can be ensured. In this embodiment, according to the reference scaling factor of the forward audio unit, the reference scaling factor of the current audio unit is processed for continuity, so that the scaling factor of the sampling point at the head of the processed current audio unit is continuous with the scaling factor of the sampling point at the tail of the forward audio unit, wherein the scaling factor of the sampling point at the tail of the forward audio unit is the reference scaling factor of the forward audio unit.

[0099] Exemplarily, the reference scaling factor of the forward audio unit is 1, and the reference scaling factor of the current audio unit is 0.6. Obviously, the reference scaling factor of the forward audio unit and the reference scaling factor of the current audio unit are jumpy and not continuous.

[0100] In one implementation, the position of the buffer area of ​​the current audio unit is determined, and the starting position of the buffer area of ​​the current audio unit is the starting position of the current audio unit; according to the baseline scaling factor of the current audio unit and the baseline scaling factor of the forward audio unit, the scaling factors of the sampling points in the buffer area are interpolated, and the scaling factors of the sampling points in the buffer area are determined according to the interpolation results.

[0101] The buffer area refers to an area located at the head of the current audio unit, and the buffer area is an area where the scaling factor interpolation is required. The length of the buffer area can be a fixed length or can be determined dynamically.

[0102] Exemplarily, the length of the buffer area of ​​the current audio unit is determined based on a first preset length of the buffer area and / or the position of the volume peak of the current audio unit, wherein the end position of the buffer area cannot exceed the position of the volume peak of the current audio unit, that is, the buffer area cannot include the position of the volume peak of the current audio unit, that is, the volume peak does not participate in the interpolation.

[0103] The end position of the buffer area is limited to not exceed the position of the volume peak of the current audio unit in order to ensure that the peak value of the sampling point in the current audio unit can be limited within the threshold in any case. When the volume peak of the forward audio unit is smaller than the volume peak of the current audio unit, that is, when the reference scaling factor of the forward audio unit is larger than the reference scaling factor of the current audio unit, the interpolation scaling factor of each sampling point after interpolation in the buffer area of ​​the current audio unit will be larger than the reference scaling factor. If the buffer area contains the sampling point corresponding to the volume peak position, the sampling point corresponding to the volume peak position will be applied with an interpolation scaling factor larger than the reference scaling factor, which will result in that the sampling point corresponding to the volume peak position must not be limited within the threshold, and the sampling points around the volume peak may also not be limited within the threshold. Therefore, the position of the volume peak cannot be included in the buffer area.

[0104] The first preset length is a fixed length, for example, 1 ms, while the position of the volume peak of the current audio unit changes dynamically, and the positions of the volume peaks of different audio units are different.

[0105] The first preset length of the buffer area and / or the position of the volume peak of the current audio unit include the following three situations:

[0106] (1) The length of the buffer area is determined only according to the first preset length. For example, when the first preset length is 1 ms, for an audio unit with a length of 10 ms, a head area with a length of 1 ms is selected as the buffer area.

[0107] (2) The length of the buffer area is determined only according to the position of the volume peak of the current audio unit. For example, the length between the start position of the current audio unit and the volume peak is used as the length of the buffer area. It can also be described as using the volume peak as the end position of the buffer area. Alternatively, 1 / 2 of the length between the start position of the current audio unit and the volume peak is used as the length of the buffer area.

[0108] (3) Determine the length of the buffer area of ​​the current audio unit according to the first preset length and the position of the volume peak of the current audio unit. Exemplarily, the minimum value of the first preset length and the position of the volume peak of the current audio unit can be taken as the length of the buffer area of ​​the current audio unit. That is, if the first audio length is less than the position of the volume peak of the current audio unit, the first audio length is taken as the length of the buffer area; if the first audio length is greater than or equal to the position of the volume peak of the current audio unit, the position of the volume peak of the current audio unit is taken as the length of the buffer area.

[0109] In this embodiment, the interpolation method is not limited, and any existing interpolation method can be used to interpolate the buffer area of ​​the current audio unit. For example, linear interpolation is used to interpolate the buffer area. Linear interpolation is one of the simplest interpolation methods, which assumes that the function changes linearly between two known points.

[0110] In this embodiment, the reference scaling factor of the forward audio unit may be greater than or less than the reference scaling factor of the current audio unit. When the reference scaling factor of the forward audio unit is greater than the reference scaling factor of the current audio unit, the interpolation scaling factor of each sampling point obtained by interpolation in the buffer area is greater than the reference scaling factor, and when the reference scaling factor of the forward audio unit is less than the reference scaling factor of the current audio unit, the interpolation scaling factor of each sampling point obtained by interpolation in the buffer area is less than the reference scaling factor.

[0111] Figure 2This is a schematic diagram of the interpolation results of the buffer area. Figure 2 As shown, assuming that the i-1th audio unit is a forward audio unit, the i-th audio unit is a current audio unit, the base scaling factor of the i-1th audio unit is 1, the base scaling factor of the i-th audio unit is 0.6, the straight line corresponding to the base scaling factor of the i-1th audio unit is discontinuous with the straight line corresponding to the base scaling factor of the i-th audio unit, that is, the scaling factor at the end of the i-1th audio unit is discontinuous with the scaling factor at the head of the i-th audio unit. In order to make the two continuous, the scaling factor of the buffer area at the head of the i-th audio unit is interpolated, and the interpolation scaling factors of each sampling point in the buffer area are all located on the straight line corresponding to the interpolation scaling factor or on both sides of the straight line corresponding to the interpolation scaling factor. The straight line corresponding to the interpolation scaling factor can be understood as a straight line fitted by the interpolation scaling factors of each sampling point in the buffer area.

[0112] In one implementation, the interpolation scaling factor of each sampling point obtained by interpolation in the buffer area may be directly used as the scaling factor of each sampling point.

[0113] In another implementation, the interpolation scaling factor of each sampling point obtained by interpolation in the buffer area is processed to obtain the scaling factor of each sampling point.

[0114] Specifically, for each sampling point in the buffer area, the ratio of the threshold of the audio limiter to the amplitude of the sampling point is calculated, and the ratio is used as the maximum scaling factor of the sampling point. The interpolation scaling factor of each sampling point is compared with the maximum scaling factor. When the interpolation scaling factor of the sampling point is less than or equal to the maximum scaling factor of the sampling point, the interpolation scaling factor of the sampling point is determined to be the scaling factor of the sampling point. When the interpolation scaling factor of the sampling point is greater than the maximum scaling factor of the sampling point, the maximum scaling factor of the sampling point is determined to be the scaling factor of the sampling point.

[0115] This is because when the interpolation scaling factor of a sampling point is greater than the maximum scaling factor of the sampling point, after the interpolation scaling factor of the sampling point is used for limiting, the amplitude of the limited sampling point will exceed the threshold of the audio limiter.

[0116] Optionally, after determining the scaling factors of the sampling points in the buffer area, the scaling factors of the sampling points in the buffer area are smoothed so that the amplitudes of the sampling points in the buffer area can smoothly transition, and ultimately the scaling factors of the sampling points in the entire current audio unit can be smooth and continuous.

[0117] S104: performing position limiting processing on each sampling point of the current audio unit according to the scaling factor of each sampling point of the current audio unit.

[0118] The current audio unit is limited by the final scaling factor obtained by the above method to obtain a limited audio unit. The limited audio unit limits each sampling point within the threshold range of the audio limiter and is continuous with other audio units. In addition, it can also ensure that the relative size relationship of the amplitude of each sampling point in the audio unit after limiting is consistent with the relative size relationship before limiting, that is, the relative relationship between the sampling points will not be lost.

[0119] The audio and video editing software performs limiting processing on each audio unit in the current audio frame in turn. When it is determined that the current audio unit is the last audio unit in the current audio frame, the limiting results of all audio units in the current audio frame are spliced ​​to obtain the limiting result of the current audio frame; and the limiting result of the current audio frame is output.

[0120] The audio and video editing software plays the current audio frame according to the limiting result of the current audio frame, and saves the limiting result of the current audio frame for subsequent playback.

[0121] In this embodiment, according to the volume and threshold of the audio limiter, the current audio frame is preprocessed to obtain at least one audio unit to be limited; at least one audio unit is processed in sequence, and the reference scaling factor of the current audio unit is determined according to the volume peak value of the current audio unit, the threshold and the buffer amplitude; according to the reference scaling factor of the forward audio unit adjacent to the current audio unit, the reference scaling factor of the current audio unit is processed for continuity to obtain the scaling factor of each sampling point of the current audio unit, and the scaling factor of the sampling point at the head of the current audio unit is continuous with the reference scaling factor of the sampling point at the tail of the forward audio unit; according to the scaling factor of each sampling point of the current audio unit, each sampling point of the current audio unit is limited. This method only limits the current audio unit according to the reference scaling factor of the forward audio unit, without the information of the backward audio unit, thereby achieving zero delay in the limiting process.

[0122] Based on the first embodiment, the second embodiment of the present application provides an audio processing method. Figure 3 This is a flowchart of the audio processing method provided in Embodiment 2 of the present application. This embodiment is described by taking audio and video editing software as an example. Figure 3 As shown, the method provided in this embodiment includes the following steps.

[0123] S201. Preprocess a current audio frame according to a volume and a threshold of an audio limiter to obtain at least one audio unit to be limited.

[0124] This step is the preprocessing stage. The audio file is input into the audio frame in sequence. The audio and video editing software extracts the original audio data from the current audio frame, scales the original audio data according to the volume or gain of the audio limiter, and obtains the audio data to be limited.

[0125] Compare the length of the current audio frame with the second preset length. When the length of the current audio frame is less than or equal to the second preset length, determine the audio data to be limited as an audio unit; when the length of the current audio frame is greater than the second preset length, divide the audio data to be limited into multiple audio units according to the second preset length.

[0126] S202: Determine a reference scaling factor of the i-th audio unit according to a volume peak value of the i-th audio unit, a threshold value of an audio limiter, and a buffer amplitude.

[0127] The audio and video editing software traverses each audio unit in turn. For the i-th audio unit, i ranges from 1 to N, and N is the number of audio units to be limited included in the current audio frame. The calculation method of the reference scaling factor of the i-th audio unit refers to the description of the first embodiment, which will not be repeated here.

[0128] Steps S203 and S206 are executed after step S202. There is no order in which S203 and S206 are executed, and they may be executed simultaneously.

[0129] S203: Determine the length of the buffer area of ​​the ith audio unit according to the first preset length and the position of the volume peak of the ith audio unit.

[0130] Exemplarily, the minimum value of the first preset length and the position of the volume peak of the current audio unit may be taken as the length of the buffer area of ​​the current audio unit, and the first preset length may be, for example, 1 ms.

[0131] Step S204 is executed after step S203.

[0132] S206: Determine whether the i-th audio unit is the first audio unit.

[0133] If the i-th audio unit is the first audio unit, step S207 is executed to update the reference scaling factor of the forward audio unit; if the i-th audio unit is not the first audio unit, no operation is performed.

[0134] S207: Update the reference scaling factor of the forward audio unit.

[0135] The first audio unit refers to the first audio unit of the entire audio file. Since the first audio unit has no forward audio unit, a reference scaling factor of the forward audio unit needs to be set for it.

[0136] In this embodiment, the reference scaling factor of the forward audio unit of the first audio unit may be determined according to the reference scaling factor of the first audio unit. Exemplarily, the reference scaling factor of the forward audio unit of the first audio unit may be obtained by adding or subtracting a preset value from the reference scaling factor of the first audio unit.

[0137] Optionally, in some implementations, a default value may be set for the reference scaling factor of the forward audio unit. For the first audio unit, the reference scaling factor of the forward audio unit is the default value.

[0138] The reference scaling factor of the forward audio unit is continuously updated. When the i-th audio unit is not the first audio unit, the reference scaling factor of the forward audio unit is the reference scaling factor calculated based on the volume peak value of the i-1-th audio unit, the threshold of the audio limiter, and the buffer amplitude.

[0139] S204: perform scaling factor interpolation on the sampling points in the buffer area of ​​the i-th audio unit according to the reference scaling factor of the i-th audio unit and the reference scaling factor of the forward audio unit, and determine the scaling factors of the sampling points in the buffer area according to the interpolation results.

[0140] The forward audio unit of the i-th audio unit is the i-1-th audio unit. In this embodiment, the scaling factors of the sampling points in the buffer area are interpolated according to the reference scaling factor of the i-th audio unit and the reference scaling factor of the forward audio unit, and the maximum scaling factor of each sampling point is calculated. The maximum scaling factor of the sampling point is equal to the ratio of the threshold of the audio limiter to the amplitude of the sampling point. Then, the interpolation scaling factor of the sampling point is compared with the maximum scaling factor. When the interpolation scaling factor of the sampling point is less than or equal to the maximum scaling factor of the sampling point, the interpolation scaling factor of the sampling point is determined to be the scaling factor of the sampling point; when the interpolation scaling factor of the sampling point is greater than the maximum scaling factor of the sampling point, the maximum scaling factor of the sampling point is determined to be the scaling factor of the sampling point. Optionally, the scaling factors of the sampling points in the buffer area can be further smoothed.

[0141] S205 . Perform position limiting processing using the scaling factor of each sampling point of the i th audio unit.

[0142] After step S205, S207 and S208 are executed, wherein S207 and S208 may be executed in parallel or sequentially, which is not limited in this embodiment.

[0143] When executing S207, the reference scaling factor of the forward audio unit is updated to the reference scaling factor of the i-th audio unit.

[0144] S208: Determine whether the i-th audio unit is the last audio unit in the current audio frame.

[0145] If the i-th audio unit is the last audio unit in the current audio frame, execute step S309; ​​if the i-th audio unit is not the last audio unit in the current audio frame, update i to i plus 1, and return to execute S202.

[0146] S209: Concatenate the position limiting results of all audio units in the current audio frame to obtain the position limiting result of the current audio frame.

[0147] S210: Output the limiting result of the current audio frame.

[0148] The process of this embodiment is a complete position limiting process of an audio frame, and the audio frames in the audio file sequentially execute the above process. Step S201 is the pre-processing process of the audio frame, S202-S208 is the position limiting process of the audio frame, and S209-S210 is the post-processing process of the audio frame.

[0149] In order to better implement the audio processing method of the embodiment of the present application, the embodiment of the present application also provides an audio processing device. Figure 4 This is a structural diagram of the audio processing device provided in Example 3 of the present application, such as Figure 4 As shown, the audio processing device 100 may include: a pre-processing module 11 , a determination module 12 , a continuity processing module 13 and a limiting module 14 .

[0150] The preprocessing module 11 is used to preprocess the current audio frame according to the volume and threshold of the audio limiter to obtain at least one audio unit to be limited.

[0151] The determination module 12 is used to process the at least one audio unit in sequence, and determine a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold value of the audio limiter, and a buffer amplitude, wherein the reference scaling factor is less than or equal to 1.

[0152] The continuity processing module 13 is used to perform continuity processing on the reference scaling factor of the current audio unit according to the reference scaling factor of the forward audio unit adjacent to the current audio unit, so as to obtain the scaling factor of each sampling point of the current audio unit, and the scaling factor of the sampling point at the head of the current audio unit is continuous with the scaling factor of the sampling point at the tail of the forward audio unit.

[0153] The limiting module 14 is used to perform limiting processing on each sampling point of the current audio unit according to the scaling factor of each sampling point of the current audio unit.

[0154] In some exemplary implementations, the determination module 12 is specifically used to: calculate the sum of the volume peak value of the current audio unit and the buffer amplitude to obtain a first amplitude; when the first amplitude is greater than or equal to the threshold of the audio limiter, calculate the ratio of the threshold of the audio limiter to the first amplitude, and use the ratio as a baseline scaling factor of the current audio unit; when the first amplitude is less than the threshold of the audio limiter, determine that the baseline scaling factor of the current audio unit is 1.

[0155] In some other exemplary implementations, the determination module 12 is specifically used to: calculate the product of the volume peak value of the current audio unit and the adjustment factor, and calculate the sum of the product and the buffer amplitude to obtain a second amplitude; when the second amplitude is greater than or equal to the threshold of the audio limiter, calculate the ratio of the threshold of the audio limiter to the second amplitude, and use the ratio as the reference scaling factor of the current audio unit; when the second amplitude is less than the threshold of the audio limiter, determine that the reference scaling factor of the current audio unit is 1.

[0156] In some further exemplary implementations, the determination module 12 is specifically used to: obtain a transformation factor through exponential operation according to the volume peak value of the current audio unit and the buffer amplitude, and the transformation factor is greater than 0; calculate the sum of the volume peak value of the current audio unit and the third amplitude to obtain a fourth amplitude, and the third amplitude is the product of the volume peak value of the current audio unit and the transformation factor; when the fourth amplitude is greater than or equal to the threshold value of the audio limiter, calculate the ratio of the threshold value of the audio limiter to the fourth amplitude, and use the ratio as the reference scaling factor of the current audio unit; when the fourth amplitude is less than the threshold value of the audio limiter, determine that the reference scaling factor of the current audio unit is 1.

[0157] In some exemplary implementations, the continuity processing module 13 is specifically used to: determine the position of the buffer area of ​​the current audio unit, the starting position of the buffer area of ​​the current audio unit being the starting position of the current audio unit; interpolate the scaling factors of the sampling points in the buffer area according to the baseline scaling factor of the current audio unit and the baseline scaling factor of the forward audio unit, and determine the scaling factors of the sampling points in the buffer area according to the interpolation results.

[0158] In some exemplary implementations, the continuity processing module 13 is specifically used to: for each sampling point in the buffer area, calculate the ratio of the threshold of the audio limiter to the amplitude of the sampling point, and the ratio is the maximum scaling factor of the sampling point; when the interpolation scaling factor of the sampling point is less than or equal to the maximum scaling factor of the sampling point, determine the interpolation scaling factor of the sampling point as the scaling factor of the sampling point; when the interpolation scaling factor of the sampling point is greater than the maximum scaling factor of the sampling point, determine the maximum scaling factor of the sampling point as the scaling factor of the sampling point.

[0159] In some exemplary implementations, the continuity processing module 13 is further configured to: after determining the scaling factors of the sampling points in the buffer area according to the interpolation results, perform smoothing on the scaling factors of the sampling points in the buffer area.

[0160] In some exemplary implementations, the continuity processing module 13 is specifically used to determine the length of the buffer area of ​​the current audio unit based on a first preset length of the buffer area and / or the position of the volume peak of the current audio unit, wherein the end position of the buffer area cannot exceed the position of the volume peak of the current audio unit.

[0161] In some exemplary implementations, the continuity processing module 13 is specifically configured to take the minimum value of the first preset length and the position of the volume peak of the current audio unit as the length of the buffer area of ​​the current audio unit.

[0162] In some exemplary implementations, the preprocessing module 11 is specifically used to: extract original audio data from the current audio frame, scale the original audio data according to the volume of the audio limiter, and obtain audio data to be limited; when the length of the current audio frame is less than or equal to a second preset length, determine the audio data to be limited as an audio unit; when the length of the current audio frame is greater than the second preset length, divide the audio data to be limited into multiple audio units according to the second preset length.

[0163] In some exemplary implementations, the device 100 also includes a post-processing module, which is used to: determine that the current audio unit is the last audio unit in the current audio frame; splice the limiting results of all audio units in the current audio frame to obtain the limiting results of the current audio frame; and output the limiting results of the current audio frame.

[0164] In some exemplary implementations, the continuity processing module 13 is further configured to: determine a reference scaling factor of a forward audio unit of the first audio unit according to the reference scaling factor of the first audio unit.

[0165] In some exemplary implementations, the buffer amplitude is associated with a threshold of the audio limiter.

[0166] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here.

[0167] The above describes the device 100 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0168] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0169] An embodiment of the present application also provides an electronic device. Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present application is shown in FIG. Figure 5 As shown, the electronic device 200 may include:

[0170] The memory 21 and the processor 22, the memory 21 is used to store the computer program and transmit the program code to the processor 22. In other words, the processor 22 can call and run the computer program from the memory 21 to implement the following method:

[0171] According to the volume and threshold of the audio limiter, the current audio frame is preprocessed to obtain at least one audio unit to be limited; the at least one audio unit is processed in sequence, and the reference scaling factor of the current audio unit is determined according to the volume peak value of the current audio unit, the threshold of the audio limiter and the buffer amplitude, and the reference scaling factor is less than or equal to 1; according to the reference scaling factor of the forward audio unit adjacent to the current audio unit, the reference scaling factor of the current audio unit is processed for continuity to obtain the scaling factor of each sampling point of the current audio unit, and the scaling factor of the sampling point at the head of the current audio unit is continuous with the scaling factor of the sampling point at the tail of the forward audio unit; according to the scaling factor of each sampling point of the current audio unit, each sampling point of the current audio unit is limited.

[0172] In some embodiments of the present application, the processor 22 may include but is not limited to: a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0173] In some embodiments of the present application, the memory 21 includes, but is not limited to: a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink DRAM (SLDRAM), and direct RAM bus random access memory (Direct Rambus RAM, DR RAM).

[0174] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 22 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 200.

[0175] like Figure 5 As shown, the electronic device 200 may further include: a transceiver 23 , which may be connected to the processor 22 or the memory 21 .

[0176] The processor 22 may control the transceiver 23 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 23 may include a transmitter and a receiver. The transceiver 23 may further include an antenna, and the number of antennas may be one or more.

[0177] In some embodiments of the present application, the electronic device 200 may also include a basic input / output system (I / O system) that helps transmit information between various devices within the computer device, and a large-capacity storage device for storing an operating system, application programs, and other program modules.

[0178] In some embodiments, the basic input / output system includes a display for displaying information and an input device such as a mouse and a keyboard for user inputting information. The display and the input device are connected to the processor 22 via an input / output controller connected to the system bus. The basic input / output system may also include an input / output controller for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller also provides output to a display screen, a printer, or other types of output devices.

[0179] Understandable, although Figure 5 Not shown, the electronic device 200 may further include more components. For example, the communication device may further include a Bluetooth module, a power module, an audio module, etc., which will not be described in detail here.

[0180] It should be understood that the various components in the electronic device 200 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0181] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0182] The present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of the electronic device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the electronic device executes the corresponding process in the above method embodiment, which will not be described here for brevity.

[0183] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0184] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0185] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An audio processing method, characterized in that: The method comprises: Preprocessing the current audio frame to obtain at least one audio unit to be limited according to the volume and threshold of the audio limiter, wherein the volume of the audio limiter represents a multiple of the audio waveform scaling, and the threshold of the audio limiter represents an output range of the audio waveform; Processing the at least one audio unit to be limited in sequence, determining a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold value of the audio limiter, and a buffer amplitude, wherein the buffer amplitude is used to increase the volume peak value of the current audio unit, and the reference scaling factor is less than or equal to 1; According to the reference scaling factor of the forward audio unit adjacent to the current audio unit, the reference scaling factor of the current audio unit is processed so that the scaling factor of the sampling point at the head of the current audio unit is continuous with the scaling factor of the sampling point at the tail of the forward audio unit, to obtain the scaling factor of each sampling point of the current audio unit; According to the scaling factor of each sampling point of the current audio unit, the sampling points of the current audio unit are limited; The preprocessing of the current audio frame to obtain at least one audio unit to be limited according to the volume and threshold of the audio limiter includes: Extracting original audio data from the current audio frame, scaling the original audio data according to the volume of the audio limiter, and determining at least one audio unit to be limited according to the audio data whose amplitude exceeds the threshold of the audio limiter after scaling.

2. The method according to claim 1, characterized in that The step of determining a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold value of the audio limiter, and a buffer amplitude includes: Calculating the sum of the volume peak value of the current audio unit and the buffer amplitude to obtain a first amplitude; When the first amplitude is greater than or equal to the threshold of the audio limiter, calculating a ratio of the threshold of the audio limiter to the first amplitude, and using the ratio as a reference scaling factor of the current audio unit; When the first amplitude is less than a threshold of the audio limiter, a reference scaling factor of the current audio unit is determined to be 1.

3. The method according to claim 1, characterized in that The step of determining a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold value of the audio limiter, and a buffer amplitude includes: Calculating the product of the volume peak value of the current audio unit and the adjustment factor, and calculating the sum of the product and the buffer amplitude to obtain a second amplitude; When the second amplitude is greater than or equal to the threshold of the audio limiter, calculating a ratio of the threshold of the audio limiter to the second amplitude, and using the ratio as a reference scaling factor of the current audio unit; When the second amplitude is less than a threshold of the audio limiter, a reference scaling factor of the current audio unit is determined to be 1.

4. The method according to claim 1, characterized in that: The step of determining a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold value of the audio limiter, and a buffer amplitude includes: According to the volume peak value of the current audio unit and the buffer amplitude, a transformation factor is obtained by exponential operation, and the transformation factor is greater than 0; Calculate the sum of the volume peak value of the current audio unit and the third amplitude to obtain a fourth amplitude, where the third amplitude is the product of the volume peak value of the current audio unit and the transformation factor; When the fourth amplitude is greater than or equal to the threshold of the audio limiter, calculating a ratio of the threshold of the audio limiter to the fourth amplitude, and using the ratio as a reference scaling factor of the current audio unit; When the fourth amplitude is less than a threshold of the audio limiter, a reference scaling factor of the current audio unit is determined to be 1.

5. The method according to claim 1, characterized in that The step of performing continuity processing on the reference scaling factor of the current audio unit according to the reference scaling factor of the forward audio unit adjacent to the current audio unit to obtain the scaling factor of each sampling point of the current audio unit includes: Determine the position of the buffer area of ​​the current audio unit, the starting position of the buffer area of ​​the current audio unit being the starting position of the current audio unit; According to the reference scaling factor of the current audio unit and the reference scaling factor of the forward audio unit, scaling factors of the sampling points in the buffer area are interpolated, and the scaling factors of the sampling points in the buffer area are determined according to the interpolation results.

6. The method according to claim 5, characterized in that The step of determining the scaling factor of the sampling points in the buffer area according to the interpolation result comprises: For each sampling point in the buffer area, calculating a ratio of a threshold of the audio limiter to an amplitude of the sampling point, wherein the ratio is a maximum scaling factor of the sampling point; When the interpolation scaling factor of the sampling point is less than or equal to the maximum scaling factor of the sampling point, determining the interpolation scaling factor of the sampling point as the scaling factor of the sampling point; When the interpolation scaling factor of the sampling point is greater than the maximum scaling factor of the sampling point, the maximum scaling factor of the sampling point is determined as the scaling factor of the sampling point.

7. The method according to claim 6, characterized in that After determining the scaling factors of the sampling points in the buffer area according to the interpolation results, the method further includes: The scaling factors of the sampling points within the buffer area are smoothed.

8. The method according to claim 5, characterized in that The determining the position of the buffer area of ​​the current audio unit includes: The length of the buffer area of ​​the current audio unit is determined according to a first preset length of the buffer area and / or the position of the volume peak of the current audio unit, wherein the end position of the buffer area cannot exceed the position of the volume peak of the current audio unit.

9. The method according to claim 8, characterized in that The determining the length of the buffer area of ​​the current audio unit according to a first preset length of the buffer area and / or a position of a volume peak of the current audio unit includes: The minimum value of the first preset length and the position of the volume peak of the current audio unit is taken as the length of the buffer area of ​​the current audio unit.

10. The method according to any one of claims 1 to 9, characterized in that The step of determining at least one audio unit to be limited according to the audio data whose amplitude exceeds the threshold of the audio limiter after scaling comprises: When the length of the current audio frame is less than or equal to a second preset length, determining the audio data whose scaled amplitude exceeds the threshold of the audio limiter as an audio unit; When the length of the current audio frame is greater than the second preset length, the audio data whose scaled amplitude exceeds the threshold of the audio limiter is divided into a plurality of audio units according to the second preset length.

11. The method according to claim 10, characterized in that After performing position limiting processing on each sampling point of the current audio unit according to the scaling factor of each sampling point of the current audio unit, the method further includes: Determine that the current audio unit is the last audio unit in the current audio frame; splicing the limiting results of all audio units in the current audio frame to obtain the limiting result of the current audio frame; Output the limiting result of the current audio frame.

12. The method according to any one of claims 1 to 9, characterized in that When the current audio unit is the first audio unit, before the base scaling factor of the current audio unit is processed for continuity according to the base scaling factor of the forward audio unit adjacent to the current audio unit, the method further includes: A reference scaling factor of a forward audio unit of the first audio unit is determined according to the reference scaling factor of the first audio unit.

13. The method according to any one of claims 1 to 9, characterized in that The buffer amplitude is associated with a threshold value of the audio limiter.

14. An audio processing device, characterized in that: include: A preprocessing module, configured to preprocess the current audio frame according to the volume and threshold of the audio limiter to obtain at least one audio unit to be limited, wherein the volume of the audio limiter represents a multiple of the audio waveform scaling, and the threshold of the audio limiter represents an output range of the audio waveform; A determination module, configured to process the at least one audio unit to be limited in sequence, and determine a reference scaling factor of the current audio unit according to a volume peak value of the current audio unit, a threshold value of the audio limiter, and a buffer amplitude, wherein the buffer amplitude is used to increase the volume peak value of the current audio unit, and the reference scaling factor is less than or equal to 1; a continuity processing module, configured to process the reference scaling factor of the current audio unit according to the reference scaling factor of the forward audio unit adjacent to the current audio unit so that the scaling factor of the sampling point at the head of the current audio unit is continuous with the scaling factor of the sampling point at the tail of the forward audio unit, and obtain the scaling factor of each sampling point of the current audio unit; A limiting module, used for performing limiting processing on each sampling point of the current audio unit according to a scaling factor of each sampling point of the current audio unit; Among them, the preprocessing module is specifically used to: extract original audio data from the current audio frame, scale the original audio data according to the volume of the audio limiter, and determine at least one audio unit to be limited based on the audio data whose amplitude after scaling exceeds the threshold of the audio limiter.

15. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 13.

Images