Audio processing method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, and discloses an audio processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining to-be-processed media data and audio metadata of the to-be-processed media data; first loudness compensation is carried out on the to-be-processed media data based on the audio metadata to obtain first media data, and the first loudness compensation comprises loudness compensation of dynamic range control; determining a second loudness compensation value corresponding to a peak limit based on an audio feature of the first media data; and performing second loudness compensation and peak limit on the first media data based on the second loudness compensation value, and determining target media data for playing.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an audio processing method, device, electronic device and storage medium. Background Art

[0002] The terminal needs to process the audio before playing the media data to ensure balanced loudness. Summary of the invention

[0003] In view of this, the present disclosure provides an audio processing method, device, electronic device and storage medium to solve the problem of loudness equalization.

[0004] In a first aspect, the present disclosure provides an audio processing method, the method comprising:

[0005] Acquire media data to be processed and audio metadata of the media data to be processed;

[0006] Performing first loudness compensation on the to-be-processed media data based on the audio metadata to obtain first media data, where the first loudness compensation includes loudness compensation of dynamic range control;

[0007] determining a second loudness compensation value corresponding to the peak limit based on the audio characteristics of the first media data;

[0008] The first media data is subjected to second loudness compensation and peak limiting based on the second loudness compensation value to determine target media data for playback.

[0009] In a second aspect, the present disclosure provides an audio processing device, the device comprising:

[0010] An acquisition module, used to acquire the media data to be processed and the audio metadata of the media data to be processed;

[0011] A first loudness processing module, configured to perform first loudness compensation on the to-be-processed media data based on the audio metadata to obtain first media data, wherein the first loudness compensation includes loudness compensation of dynamic range control;

[0012] A second loudness compensation value determining module, configured to determine a second loudness compensation value corresponding to a peak value limitation based on an audio feature of the first media data;

[0013] The second loudness processing module is configured to perform second loudness compensation and peak limiting on the first media data based on the second loudness compensation value, and determine target media data for playback.

[0014] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the audio processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the audio processing method of the first aspect or any corresponding embodiment thereof.

[0016] The audio processing method provided by the embodiment of the present disclosure has a certain impact on the loudness after the peak limiting processing of the media data, and the dynamic range control also has an impact on the loudness. Therefore, the dynamic range control and the peak limiting will perform loudness compensation respectively. Among them, the second loudness compensation value used for peak limiting is determined based on the audio characteristics of the first media data, which is related to the first media data after the dynamic range control processing, and can ensure the accuracy of the obtained second loudness compensation value. The loudness compensation of this method exists in the dynamic range control and the peak limiting, so that the loudness error of the target media data can be controlled within a certain range, thereby improving the loudness equalization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a flowchart of an audio processing method according to an embodiment of the present disclosure;

[0019] Figure 2 is a flowchart of another audio processing method according to an embodiment of the present disclosure;

[0020] Figure 3 is a flowchart of another audio processing method according to an embodiment of the present disclosure;

[0021] Figure 4 is a data flow block diagram of audio processing according to an embodiment of the present disclosure;

[0022] Figure 5 is a data flow block diagram of audio processing according to an embodiment of the present disclosure;

[0023] Figure 6is a structural block diagram of an audio processing device according to an embodiment of the present disclosure;

[0024] Figure 7 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0026] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0027] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0028] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] In the related art, in order to achieve loudness balance at the playback end, different videos with different loudness are pulled to the same release loudness or playback loudness by multiplying by volume gain, processing by dynamic range control curve, and peak limiting. However, in the above scheme, after the dynamic range control curve and peak limit processing, the loudness will be affected to a certain extent. Therefore, even after the volume gain processing, it will make it difficult to achieve a balanced loudness effect for the media data used for playback.

[0031] Based on this, the audio processing method provided by the embodiment of the present disclosure introduces a loudness compensation value during dynamic range control and peak limit processing to compensate for the influence of dynamic range control and peak limit processing on loudness, thereby achieving the effect of loudness balance. Therefore, the method can ensure that the loudness of the target media data for playback is within the error range.

[0032] Furthermore, when determining the loudness compensation value, a corresponding method for determining the loudness compensation value is selected according to the target performance requirements to balance the computing power of the playback terminal and the loudness equalization effect. That is, for a playback terminal with low computing power, a method with low processing complexity can be selected to obtain the loudness compensation value; for a playback terminal with high computing power, a method with high processing complexity can be selected to obtain the loudness compensation value.

[0033] Furthermore, since peak limiting is the last step of audio processing, in order to control the error range, different determination methods are set for different error ranges for determining the loudness compensation value for peak limiting.

[0034] Furthermore, the method is also used to obtain the frequency response curve and / or cutoff frequency of the playback device to determine the corresponding target loudness and / or loudness compensation based on the actual performance data of the playback device. Therefore, this method can provide the best loudness equalization effect for a specific device that is adapted to the playback frequency response of the device.

[0035] It should be noted that the audio processing method provided by the present disclosure can be applicable to short video playback scenarios, local media data playback scenarios, etc. There is no specific restriction on its application scenarios, and it can be set according to actual needs.

[0036] According to an embodiment of the present disclosure, an audio processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] In this embodiment, an audio processing method is provided, which can be used in electronic devices such as servers, playback terminals, etc. Figure 1 is a flowchart of an audio processing method according to an embodiment of the present disclosure, such as Figure 1 As shown, the process includes the following steps:

[0038] Step S101: acquiring media data to be processed and audio metadata of the media data to be processed.

[0039] The method for obtaining the media data to be processed is related to the application scenario of the audio processing method. If the method is applied to the short video playback scenario, the media data to be processed comes from the server corresponding to the short video playback application; if the method is applied to the local media data playback scenario, the media data to be processed comes from the locally cached media data. There is no limitation on the method for obtaining the media data to be processed, which is set according to actual needs.

[0040] The audio metadata of the media data to be processed includes but is not limited to the source loudness, short-time maximum loudness, energy peak, loudness range, voice ratio, loudness range start and end points, and device loudness compensation of the media data to be processed. These audio metadata are obtained by analyzing the audio features of the media data to be processed, and the analysis method is not limited here. If the application scenario is a terminal video playback scenario, the audio metadata is sent to the playback terminal together with the media data to be processed; if the application scenario is a local media data playback scenario, the audio metadata is obtained by analyzing the local media data to be processed.

[0041] Step S102: Perform first loudness compensation on the media data to be processed based on the audio metadata to obtain first media data.

[0042] The first loudness compensation includes loudness compensation of dynamic range control.

[0043] The audio metadata includes the source loudness of the media data to be played, and the target loudness is determined by combining other data in the audio metadata. The loudness gain is obtained by the difference between the target loudness and the source loudness, and the loudness gain is used to perform loudness compensation processing on the media data to be processed. Therefore, the first loudness compensation includes the processing of the loudness gain and the loudness compensation of the dynamic range control.

[0044] The loudness compensation of the dynamic range control is determined based on the audio metadata and the control parameters of the dynamic range control, that is, the loudness compensation value is obtained by determining the influence of the dynamic range control on the loudness and then combining the audio metadata.

[0045] Based on this, the media data to be processed is processed by loudness gain, and then loudness compensation of dynamic range control is performed on it using the control parameters of dynamic range control to obtain the first media data. Of course, other processing can also be performed before dynamic range control, which is not limited here.

[0046] Step S103: determining a second loudness compensation value corresponding to the peak value limitation based on the audio feature of the first media data.

[0047] The audio features of the first media data include but are not limited to the audio peak of the first media data. Since the peak limit is used to limit the audio peak of the first media data, the influence of the peak limit on the loudness can be determined by combining the audio peak before the peak limit and the audio peak after the peak limit to obtain the second loudness compensation value. Of course, the second loudness compensation value can also be determined by a loudness compensation value model obtained by pre-training. The input of the loudness compensation value model includes the audio features of the first media data, and the output includes the second loudness compensation value. Among them, since the loudness compensation value is used to compensate for the influence of the peak limit processing on the loudness, the input of the loudness compensation value model also includes the processing parameters of the peak limit. Furthermore, since the first media data is obtained after the dynamic range control processing, the input of the loudness compensation value model also includes the control parameters of the dynamic range control, and so on. The input parameters of the loudness compensation value model can be set according to actual needs.

[0048] Step S104: performing second loudness compensation and peak limiting on the first media data based on the second loudness compensation value, and determining target media data for playback.

[0049] After obtaining the second loudness compensation value, the first media data is subjected to peak limiting processing, and then the second loudness compensation value is used to perform loudness compensation on the media data subjected to peak limiting processing, thereby obtaining target media data for playback. Alternatively, the second loudness compensation value may be used to perform second loudness compensation on the first media data, and then peak limiting is performed on the first media data to obtain target media data.

[0050] In the audio processing method provided in this embodiment, since the processing of media data through peak limiting will have a certain impact on loudness, and dynamic range control will also have an impact on loudness, therefore, dynamic range control and peak limiting will perform loudness compensation respectively. Among them, the second loudness compensation value used for peak limiting is determined based on the audio characteristics of the first media data, which is related to the first media data after dynamic range control processing, and can ensure the accuracy of the obtained second loudness compensation value. The loudness compensation of this method exists in dynamic range control and peak limiting, so that the loudness error of the target media data can be controlled within a certain range, thereby improving the loudness equalization effect.

[0051] In this embodiment, an audio processing method is provided, which can be used in electronic devices such as servers, playback terminals, etc. Figure 2 is a flowchart of an audio processing method according to an embodiment of the present disclosure, such as Figure 2 As shown, the process includes the following steps:

[0052] Step S201: Acquire the media data to be processed and the audio metadata of the media data to be processed. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0053] Step S202: Perform first loudness compensation on the media data to be processed based on the audio metadata to obtain first media data.

[0054] The first loudness compensation includes loudness compensation of dynamic range control. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0055] Step S203: determining a second loudness compensation value corresponding to the peak value limitation based on the audio feature of the first media data.

[0056] Specifically, the above step S203 includes:

[0057] Step S2031, obtaining target performance requirements.

[0058] The target performance requirement is used to characterize the demand for audio processing results, focusing on processing performance or loudness equalization. The target performance requirement can be determined by querying the computing power of the playback device. If the computing power is lower than the preset value, it means that the processing performance of the playback device is insufficient, and audio processing can be performed from the perspective of saving performance overhead. If the computing power is higher than the preset value, it means that the processing performance of the playback device is good, and audio processing can be performed from the perspective of improving the loudness equalization effect.

[0059] The target performance requirement may also be determined in an interactive manner, for example, by displaying an interactive control of the performance requirement on the interface of the playback terminal. The user interacts with the interactive control, and accordingly, the user's interactive operation result is obtained to determine the target performance requirement.

[0060] The target performance requirement can also be represented by an error range. For example, an input control for the error range is set on the interface of the playback terminal. The user interacts with the input control and obtains the error range value input by the user accordingly. The target performance requirement is determined based on the input error range value. If the error range exceeds the preset value, it means that the current focus is on processing performance; if the error range is lower than the preset value, it means that the current focus is on the effect of loudness equalization.

[0061] In some optional implementations, the target performance requirement is determined based on the loudness processing mode of the current playback device or the error range of loudness compensation. The loudness processing mode includes at least two loudness processing sub-modes, for example, loudness processing sub-mode 1 indicates that it focuses on processing performance, and loudness processing sub-mode 2 indicates that it focuses on the effect of loudness equalization. The loudness processing mode can be determined in an interactive manner, for example, the current playback device provides an interactive interface, which is used to display at least two loudness processing sub-modes. By interacting with the interactive interface, the target loudness processing sub-mode is determined, thereby obtaining the target performance requirement.

[0062] The target performance requirement is determined by the loudness processing mode or the error range of loudness compensation, that is, multiple ways are provided to determine the target performance requirement, thereby expanding the scenarios of loudness equalization processing.

[0063] Step S2032: Determine a method for determining a second loudness compensation value corresponding to the target performance requirement based on a correspondence between the performance requirement and the method for determining a loudness compensation value corresponding to the peak limit.

[0064] Different performance requirements correspond to different loudness compensation value determination methods, where the loudness compensation value is used to compensate for the impact on loudness caused by the peak limiting process. For example, performance requirement a corresponds to loudness compensation value determination method a1, performance requirement b corresponds to loudness compensation value determination method b1, and so on.

[0065] After the target performance requirement is obtained, the method for determining the second loudness compensation value corresponding to the target performance requirement is obtained by querying the above correspondence relationship, wherein the second loudness compensation value is a loudness compensation value corresponding to the peak limit.

[0066] In the above description, the target performance requirement can be obtained based on the loudness processing mode of the current playback device. If the selected loudness processing mode represents that the target performance requirement is focused on processing performance, the processing complexity of the method for determining the second loudness compensation value is relatively low; if the selected loudness processing mode represents that the target performance requirement is focused on the effect of loudness equalization, the processing complexity of the method for determining the second loudness compensation value is relatively high.

[0067] In some optional implementations, the loudness equalization effect in the performance requirement is positively correlated with the processing complexity of the method for determining the loudness compensation value of peak limitation, and the processing performance of the loudness equalization in the performance requirement is negatively correlated with the processing complexity of the method for determining the loudness compensation value of peak limitation.

[0068] Specifically, the higher the processing complexity of the loudness compensation value determination method for peak limiting, the better the loudness equalization effect; vice versa. The lower the processing complexity of the loudness compensation value determination method for peak limiting, the lower the requirement for processing performance.

[0069] For example, if the computing power of the current playback terminal is limited, it is necessary to save processing overhead to meet higher processing performance. Accordingly, the processing complexity of the corresponding loudness compensation value determination method is relatively low.

[0070] In the performance requirement, the loudness equalization effect is positively correlated with the processing complexity, and in the performance requirement, the processing performance of the loudness equalization is negatively correlated with the processing complexity, thereby satisfying the balance between the device processing performance and the equalization effect of the playback device.

[0071] In some optional implementations, the peak loudness compensation value determination method includes a determination method based on a mapping relationship between the peak loudness compensation value and the audio feature and a determination method based on a peak loudness compensation model, the input of the peak loudness compensation model includes the audio feature, and the output includes the loudness compensation value. The processing complexity of the determination method based on the mapping relationship between the peak loudness compensation value and the audio feature is less than the processing complexity of the determination method based on the peak loudness compensation model.

[0072] There are two methods for determining the peak loudness compensation value. The processing complexity and loudness equalization effect of the determination method based on the mapping relationship are both lower than those of the determination method based on the peak loudness compensation model. The mapping relationship corresponds to the peak loudness compensation value and the audio feature. The peak loudness compensation value and the audio feature can be curve-fitted to obtain a fitting curve, and the mapping relationship between the peak loudness compensation value and the audio feature can be represented by the fitting curve.

[0073] The input of the peak loudness compensation model includes audio features, and the output includes loudness compensation values. The initial peak loudness compensation model is trained by the sample data set, and the model parameters of the initial peak loudness compensation model are fixed after multiple rounds of iterations, thereby obtaining the peak loudness compensation model. The audio features include but are not limited to the audio peak and source loudness of the first media data, and the input of the peak loudness compensation model may also include control parameters of dynamic range control, such as the starting point of the dynamic range control curve and the compression ratio, etc.

[0074] By providing two different determination methods, the processing complexity requirements of different determination methods are met. The higher the processing complexity, the higher the loudness equalization effect; the lower the processing complexity, the lower the performance requirements for loudness equalization, so as to meet higher performance requirements.

[0075] Step S2033: Determine a second loudness compensation value based on the method for determining the second loudness compensation value and the audio features of the first media data.

[0076] After obtaining the method for determining the second loudness compensation value, the second loudness compensation value is obtained in combination with the audio features of the first media data. If the determination method is based on a mapping relationship, the second loudness compensation value can be calculated using the audio features and the mapping relationship; if the determination method is based on a peak loudness compensation model, the audio features are input into the peak loudness compensation model to obtain the second loudness compensation value.

[0077] In some optional implementations, if the peak loudness compensation value determination method is a determination method based on a mapping relationship between the peak loudness compensation value and the audio feature, the above step S2033 includes:

[0078] Step a1, obtaining a target peak value of the peak limit and a current audio peak value in the audio feature, and obtaining a difference between the current audio peak value and the target peak value.

[0079] Step a2: determining a second loudness compensation value based on the difference and the mapping relationship.

[0080] The mapping relationship represents the relationship between the peak difference and the loudness compensation value, wherein the peak difference is the difference between the target peak value of the peak limit and the current audio peak value in the audio feature. The relationship between the peak difference and the loudness compensation value is represented by an exponential curve, which is obtained by curve fitting of a large number of peak differences and loudness compensation values.

[0081] For example, the following formula is used to express the relationship between the second loudness compensation value and the peak value:

[0082] Second loudness compensation value = (a-log b (target peak value - current audio peak value))*c

[0083] Wherein, a, b and c are all constants greater than zero, a is greater than or equal to 1, b is greater than zero and less than 1, and c is greater than 1. For example, a=1, b=0.9, c=6.

[0084] It should be noted that if the second loudness compensation value obtained after calculation by the above formula is less than zero, the second loudness compensation value is adjusted to zero.

[0085] In the case of a determination method based on a mapping relationship, the second loudness compensation value is determined by the difference between the target peak value and the current audio peak value. The data processing volume is small and the processing performance requirement is low, so as to meet the high processing performance requirements of playback devices with limited computing power.

[0086] In some optional implementations, if the peak loudness compensation value is determined based on a peak loudness compensation model, the step S2033 includes:

[0087] Step b1, based on the loudness equalization effect in the target performance requirement, determining a target sub-model in the peak loudness compensation model, the peak loudness compensation model includes multiple sub-models, and the processing complexity of each sub-model is different.

[0088] Step b2: determining a second loudness compensation value based on the target sub-model and the audio features of the first media data.

[0089] The peak loudness compensation model includes multiple sub-models, and the processing complexity of each sub-model is different to meet different loudness equalization effects. For example, if the error range is required to be less than 5 in the loudness equalization effect, the peak loudness compensation model is determined to be used. Furthermore, different sub-models are used according to different error ranges. The smaller the error range, the higher the processing complexity of the sub-model, and accordingly, the better the loudness equalization effect obtained.

[0090] In some embodiments, the sub-model includes but is not limited to a decision tree model, a neural network model, etc. Since the processing complexity of the decision tree model is lower than that of the neural network model, the decision tree model can be used when the error range is 3 or 4; and the neural network model can be used when the error range is 1 or 2.

[0091] Of course, the above are just some examples of models, which do not limit the protection scope of the present disclosure, and are set according to actual needs.

[0092] The peak loudness compensation gain model is used to calculate the second loudness compensation value, and the second loudness compensation value is obtained by model reasoning, which can fully utilize the processing performance of the playback device to obtain a more accurate second loudness compensation value. At the same time, the peak loudness compensation gain model includes multiple sub-models, and the processing complexity of each sub-model is different to meet the needs of playback devices with different computing power.

[0093] Step S204: Perform second loudness compensation and peak limit on the first media data based on the second loudness compensation value to determine target media data for playback. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0094] In the audio processing method provided in this embodiment, different performance requirements correspond to different loudness compensation value determination methods of peak limiting, thereby meeting different target performance requirements and realizing personalized setting of loudness equalization.

[0095] In this embodiment, an audio processing method is provided, which can be used in electronic devices such as servers, playback terminals, etc. Figure 3 is a flowchart of an audio processing method according to an embodiment of the present disclosure, such as Figure 3 As shown, the process includes the following steps:

[0096] Step S301: Acquire the media data to be processed and the audio metadata of the media data to be processed. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0097] Step S302: Perform first loudness compensation on the media data to be processed based on the audio metadata to obtain first media data.

[0098] The first loudness compensation includes loudness compensation of dynamic range control.

[0099] Specifically, the above step S302 includes:

[0100] Step S3021, obtaining target performance requirements.

[0101] See above for how to obtain target performance requirements. Figure 2 The description of step S2031 of the illustrated embodiment will not be repeated here.

[0102] After the target performance requirement is determined, the method for determining the target loudness compensation value for dynamic range control and the method for determining the loudness compensation value for peak limiting are also determined accordingly.

[0103] Step S3022: Based on the correspondence between the performance requirement and the loudness compensation value determination method for dynamic range control, a target loudness compensation value determination method for dynamic range control corresponding to the target performance requirement is determined.

[0104] Corresponding to the method for determining the second loudness compensation value in the above text, the method for determining the loudness compensation value for dynamic range control is also obtained through a corresponding relationship, and different performance requirements correspond to different determination methods.

[0105] In some optional implementations, the loudness equalization effect in the performance requirement is positively correlated with the processing complexity of the method for determining the loudness compensation value of the dynamic range control, and the processing performance of the loudness equalization in the performance requirement is negatively correlated with the processing complexity of the method for determining the loudness compensation value of the dynamic range control.

[0106] For example, for playback terminals with higher computing power, emphasis is placed on loudness equalization effects, and a method for determining the loudness compensation value using dynamic range control with higher processing complexity is adopted; for playback terminals with lower computing power, due to computing power limitations, a method for determining a higher processing complexity cannot be run. Therefore, a method for determining a higher processing complexity is adopted to meet the requirements of high processing performance.

[0107] The loudness equalization effect is positively correlated with the processing complexity, and the processing performance of loudness equalization is negatively correlated with the processing complexity. By limiting the correlation relationship, it can adapt to the needs of playback devices with different computing power and different loudness equalization effects.

[0108] Step S3023: Determine a first loudness compensation value for dynamic range control based on the target loudness compensation value determination method and the audio metadata.

[0109] The target loudness compensation value determination method may include a network model with high processing complexity, or a calculation method with low processing complexity, etc. After obtaining the target loudness compensation value determination method, the first loudness compensation value for dynamic range control is obtained in combination with the audio metadata. If the network model method is adopted, the audio metadata is input into the network model to obtain the first loudness compensation value; if the calculation method is adopted, the audio metadata is used to perform corresponding calculations to obtain the first loudness compensation value.

[0110] In some optional implementations, the performance requirement includes a first level and a second level of loudness equalization effect, and the first level is lower than the second level. That is, the loudness equalization effect of the first level is lower than the loudness equalization effect of the second level, and accordingly, the processing complexity of the determination method corresponding to the first level of loudness equalization effect is lower than the determination method corresponding to the second level of loudness equalization. If the target performance requirement includes the first level of loudness equalization effect, the above step S3023 includes:

[0111] Step c1: if the length of the media data to be processed is greater than a preset length, the slope in the dynamic range control parameter and the starting point of the dynamic range in the audio metadata are obtained.

[0112] Step c2: performing loudness estimation based on the target loudness, the slope and the starting point to determine the estimated loudness.

[0113] Step c3: determining a first loudness compensation value based on a difference between the target loudness and the estimated loudness.

[0114] Step c4: if the length of the media data to be processed is less than or equal to the preset length, the maximum short-time loudness in the audio metadata is obtained.

[0115] Step c5: determining a first loudness compensation value based on the difference between the maximum short-term loudness and the target loudness.

[0116] When the length of the media data to be processed is greater than the preset length, the slope of the dynamic range control curve is obtained, and the starting point of the dynamic range is obtained from the audio features. Loudness estimation is performed based on the target loudness, the slope, and the starting point, that is, estimated loudness=(target loudness-starting point of the dynamic range) / slope+starting point of the dynamic range. After the estimated loudness is obtained, the difference between the target loudness and the estimated loudness is calculated to obtain a first loudness compensation value. For example, the first loudness compensation value=target loudness-estimated loudness+1.

[0117] When the length of the media data to be processed is less than the preset length, the difference between the maximum short-term loudness and the target loudness is calculated, and the difference may be directly determined as the first loudness compensation value, or the difference may be combined with other parameters to obtain the first loudness compensation value.

[0118] For example, when the length of the media data to be processed is less than the preset length, if the difference between the maximum short-term loudness and the target loudness is greater than the preset value, the first loudness compensation value = the maximum short-term loudness - the target loudness - the preset value; otherwise, the first loudness compensation value is set to zero. The preset value is set according to actual needs, for example, the preset value is 5. Of course, other values ​​can also be used, which are set according to actual needs.

[0119] When the length of the media data to be played is greater than the preset length, since the dynamic range has been processed before, the loudness estimation is performed here in combination with the starting point and slope of the dynamic range, so that the accuracy of the obtained loudness estimation can be guaranteed. When the length of the media data to be played is less than the preset length, the processing of the first loudness compensation value is simplified, and the first loudness compensation value is directly determined by using the difference between the maximum short-time limit and the target loudness, thereby improving the processing efficiency.

[0120] In some optional implementations, if the target performance requirement includes a second level of loudness equalization effect, and the loudness equalization effect requirement is relatively high, then a determination method with relatively high processing complexity is adopted. Based on this, the above step S3023 includes:

[0121] Step d1: if the length of the media data to be processed is greater than a preset length, a first loudness compensation value is determined based on a first loudness compensation model for dynamic range control and audio metadata.

[0122] Step d2: if the length of the media data to be processed is less than or equal to the preset length, a first loudness compensation value is determined based on a second loudness compensation model for dynamic range control and audio metadata.

[0123] The first loudness compensation model and the second loudness compensation model may be the same or different. For example, when the length of the media data to be processed is greater than the preset length, the data processing amount corresponding to the longer length is larger. Therefore, in order to simultaneously meet the processing performance and the loudness equalization effect, a model with moderate processing complexity may be used; when the length is less than or equal to the preset length, the data processing amount is smaller, and a model with higher processing complexity may be used. Therefore, the processing complexity of the first loudness compensation model is less than the processing complexity of the second loudness compensation model.

[0124] The first loudness compensation model and the second loudness compensation model may be implemented by using a decision tree model or other neural network models, and no limitation is made to their specific network structures.

[0125] Since the obtained first loudness compensation value is used to compensate for the loudness of the dynamic range control, the input of the first loudness compensation model and the second loudness compensation model may also include control parameters of the dynamic range control in addition to the audio metadata.

[0126] In the case of high requirements for loudness equalization effect, a loudness compensation model of dynamic range control is used to determine the loudness compensation value, which improves the accuracy of the loudness compensation value and further improves the effect of loudness error control.

[0127] Step S3024: Perform first loudness compensation on the media data to be processed based on the first loudness compensation value to obtain first media data.

[0128] After obtaining the first loudness compensation value, the media data to be processed may be first subjected to dynamic range control processing, and then the first loudness compensation may be performed in combination with the first loudness compensation value; or the first loudness compensation may be first performed using the first loudness compensation value, and then the dynamic range control processing may be performed. After the dynamic range processing and the first loudness compensation, the first media data is obtained.

[0129] Step S303: Determine a second loudness compensation value corresponding to the peak limit based on the audio feature of the first media data. Figure 2 Step S203 of the illustrated embodiment will not be described in detail here.

[0130] Step S304: Perform second loudness compensation and peak limit on the first media data based on the second loudness compensation value to determine target media data for playback. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0131] The audio processing method provided in this embodiment uses the correspondence between the performance requirement and the method for determining the loudness compensation value of the dynamic range control to obtain a target loudness compensation value determination method suitable for the target performance requirement during loudness compensation of the dynamic range control, so as to balance the computing power of the playback device and the loudness equalization effect.

[0132] In some optional implementations, obtaining audio metadata of the media data to be processed in the above embodiment includes: obtaining a frequency response curve of a target playback device, the frequency response curve being used to determine a target loudness. Since the frequency response curve corresponds to the target playback device, the obtained target loudness is also applicable to the target playback device.

[0133] In some optional implementations, acquiring audio metadata of the media data to be processed in the above embodiment includes: acquiring a cutoff frequency of a target playback device, where the cutoff frequency is used to determine a filtering loudness compensation value corresponding to signal energy of data to be filtered in the media data to be processed, where the data to be filtered is media data having a frequency lower than the cutoff frequency in the media data to be processed, and the target playback device is used to perform loudness processing on the media data to be processed based on the filtering loudness compensation value after filtering the data to be filtered from the media data to be processed, and the audio metadata includes a target loudness and a filtering loudness compensation value.

[0134] In the speaker playback mode, the playback terminal is limited by the cutoff frequency of the speaker, which causes the playback terminal to have a large attenuation of low-frequency signals when using the speaker to play media data. Among them, the low-frequency signal is a signal with a frequency lower than the cutoff frequency. The preset frequency can be 100Hz, 150Hz or 200Hz, which is related to the performance of the playback terminal. The low-frequency signal may be attenuated by 20dB to 40dB, or even a larger range of dB values. For a large number of short video playback scenarios such as voice, music, and movies, low-frequency signals are an important part that cannot be ignored. For example, the lack of playback energy of human voices, drums or musical instruments, etc., will inevitably lead to uneven video playback loudness and fluctuating sound when the speaker is played back after loudness equalization in the digital domain, resulting in more frequent volume adjustments in speaker mode. Since audio data below the cutoff frequency cannot be played back normally in speaker mode, this part of the data is called data to be filtered. After the playback terminal filters this part of the data to be filtered, the energy of this part of the data is compensated using the filtered loudness compensation value, and the loudness is balanced using the loudness gain.

[0135] For example, Figure 4As shown, the entire audio processing process can be divided into a parameter preparation stage and a streaming processing stage. The parameter preparation stage can be completed on the server side or on the playback terminal, and there is no limitation on it here. The parameter preparation stage includes calculating the playback target loudness and loudness gain according to the audio device selection, wherein the loudness gain is calculated by the difference between the source loudness and the target loudness. The determination of the target loudness is related to the loudness requirement information, environmental information, the playback capability of the device, and the frequency response curve of the device. The loudness requirement information includes but is not limited to the user's demand for playback loudness, and the environmental information includes whether the current environment is a quiet scene or a noisy scene, etc. In the parameter preparation stage, it is also necessary to calculate the dynamic range control curve, the above-mentioned first loudness compensation value and the second loudness compensation value. Among them, the first loudness compensation value is used for loudness compensation of dynamic range control, and the second loudness compensation value is used for loudness compensation of peak limiting. The streaming stage is to perform loudness equalization processing on the media data to be processed through the intelligent loudness equalization module, including but not limited to using the cutoff frequency of the device to filter out the signal that the device cannot play back when the filtering function is turned on, and then using the filtered loudness compensation value to perform loudness compensation processing on the filtered signal, and then through dynamic range control and peak limit, to obtain the target media data for playback.

[0136] Accordingly, Figure 5 The functional modules involved in the audio processing process are shown, including a parameter preparation module and an intelligent loudness equalization module. The parameter preparation module is used to determine the audio metadata, and the intelligent loudness equalization module is used to perform loudness equalization processing on the media data to be processed to obtain the target media data. Figure 4 and Figure 5 It describes the audio processing process from the perspectives of data flow and functional modules.

[0137] By providing an interface for obtaining a frequency response curve or a cutoff frequency, the target loudness and the filtered loudness compensation value can be accurately determined, thereby achieving the effect of performing loudness equalization processing for a specific device.

[0138] As a specific application embodiment of the embodiment of the present disclosure, it is applied to the short video playback scenario. In the short video playback scenario, the user publishes the video to the cloud. When publishing or the cloud server parameter preparation module obtains the audio metadata corresponding to each media data to be processed through processing, and sends it to the playback terminal during playback. A loudness processing mode or error range setting interface is provided on the playback terminal, and the target performance requirement is determined by interacting with the setting interface; or, the playback terminal determines the target performance requirement by querying its own device performance. The device playback capability can be predicted by volume embedding, the loudness requirement information is selected by the strategy issued by different business scenarios or selected by the user through interaction, the environmental information is obtained by collecting the current environmental volume, etc., and these data are input into the intelligent loudness equalization module for processing to obtain the target media data. Specifically, for the media data to be processed, when the playback terminal is in the speaker playback mode, it is first filtered based on the cutoff frequency of the playback terminal to filter out the signal that cannot be played back. Then the filtered loudness compensation value is calculated based on the filtered signal energy, and the filtered signal is compensated for loudness. The dynamic range control curve calculated is used to perform dynamic range control on the filtered signal, and the first loudness compensation value is used to perform first loudness compensation on the dynamic range controlled signal. The second loudness compensation value is used to perform second loudness compensation on the peak-limited signal, thereby obtaining target media data for playback.

[0139] As another specific application embodiment of the embodiment of the present disclosure, it is applied to the user playback scenario of the music playback platform. Its specific processing process is similar to the processing process applied to the short video playback scenario mentioned above, and will not be repeated here.

[0140] As another specific application embodiment of the embodiment of the present disclosure, it is applied to a local audio playback scenario. Among them, the frequency response curve and cutoff frequency of the playback device are read from the local configuration file, and the audio metadata is scanned and obtained after the local audio is archived. The loudness processing mode or error range is determined interactively to determine the target performance requirements. The device playback capability is read from the local configuration file, the loudness requirement information is selected by the policy issued by different business scenarios or selected by the user through interaction, the environmental information is obtained by collecting the current environmental volume, etc. These data are input into the intelligent loudness equalization module for processing to obtain the target media data.

[0141] The above-mentioned entire method chain of audio processing standardizes and improves the setting of input parameters, mainly realizes an intelligent loudness equalization sound effect chain that adapts to the device, and can select different sub-schemes through input parameters, that is, by determining the target performance requirements and adopting different determination methods to obtain the first loudness compensation value and the second loudness compensation value, thereby achieving a balance and providing options between saving computing performance and obtaining a more balanced loudness effect to meet different needs.

[0142] In this embodiment, an audio processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0143] This embodiment provides an audio processing device, such as Figure 6 As shown, including:

[0144] The acquisition module 601 is used to acquire the media data to be processed and the audio metadata of the media data to be processed.

[0145] The first loudness processing module 602 is configured to perform first loudness compensation on the media data to be processed based on the audio metadata to obtain first media data, where the first loudness compensation includes loudness compensation of dynamic range control.

[0146] The second loudness compensation value determining module 603 is configured to determine a second loudness compensation value corresponding to the peak value limitation based on the audio feature of the first media data.

[0147] The second loudness processing module 604 is configured to perform second loudness compensation and peak limiting on the first media data based on the second loudness compensation value, and determine target media data for playback.

[0148] In some optional implementations, the second loudness compensation value determining module 603 includes:

[0149] The first performance requirement acquisition unit is used to acquire a target performance requirement.

[0150] The second loudness compensation determining unit is configured to determine a method for determining a second loudness compensation value corresponding to a target performance requirement based on a correspondence between the performance requirement and a method for determining a loudness compensation value corresponding to a peak limit.

[0151] The second loudness compensation value determining unit is configured to determine the second loudness compensation value based on a determination method of the second loudness compensation value and an audio feature of the first media data.

[0152] In some optional implementations, the loudness equalization effect in the performance requirement is positively correlated with the processing complexity of the method for determining the loudness compensation value of peak limitation, and the processing performance of the loudness equalization in the performance requirement is negatively correlated with the processing complexity of the method for determining the loudness compensation value of peak limitation.

[0153] In some optional implementations, the peak loudness compensation value determining method includes a determining method based on a mapping relationship between the peak loudness compensation value and the audio feature and a determining method based on a peak loudness compensation model, the input of the peak loudness compensation model includes the audio feature, and the output includes the loudness compensation value; wherein the processing complexity of the determining method based on the mapping relationship between the peak loudness compensation value and the audio feature is less than the processing complexity of the determining method based on the peak loudness compensation model.

[0154] In some optional implementations, if the peak loudness compensation value determination method is a determination method based on a mapping relationship between the peak loudness compensation value and the audio feature, the second loudness compensation value determination unit includes:

[0155] The peak value acquisition subunit is used to acquire the target peak value of the peak limit and the current audio peak value in the audio feature, and obtain the difference between the current audio peak value and the target peak value.

[0156] The second loudness compensation value mapping relationship determining subunit is used to determine the second loudness compensation value based on the difference value and the mapping relationship.

[0157] In some optional implementations, if the peak loudness compensation value determination method is a determination method based on a peak loudness compensation model, the second loudness compensation value determination unit includes:

[0158] The target sub-model determining sub-unit is used to determine the target sub-model in the peak loudness compensation model based on the loudness equalization effect in the target performance requirement. The peak loudness compensation model includes multiple sub-models, and the processing complexity of each sub-model is different.

[0159] The model determination subunit of the second loudness compensation value is used to determine the second loudness compensation value based on the target sub-model and the audio features of the first media data.

[0160] In some optional implementations, the first loudness processing module 602 includes:

[0161] The second performance requirement acquisition unit is used to acquire the target performance requirement.

[0162] The target loudness compensation determining unit is configured to determine a target loudness compensation value determining method for dynamic range control corresponding to a target performance requirement based on a correspondence between the performance requirement and the loudness compensation value determining method for dynamic range control.

[0163] The first loudness compensation value determining unit is configured to determine a first loudness compensation value for dynamic range control based on a target loudness compensation value determining method and audio metadata.

[0164] The first loudness compensation unit is configured to perform first loudness compensation on the media data to be processed based on a first loudness compensation value to obtain first media data.

[0165] In some optional implementations, the loudness equalization effect in the performance requirement is positively correlated with the processing complexity of the method for determining the loudness compensation value of the dynamic range control, and the processing performance of the loudness equalization in the performance requirement is negatively correlated with the processing complexity of the method for determining the loudness compensation value of the dynamic range control.

[0166] In some optional implementations, the performance requirement includes a first level and a second level of loudness equalization effect, the first level is lower than the second level, and if the target performance requirement includes the first level of loudness equalization effect, the first loudness compensation value determination unit includes:

[0167] The data acquisition subunit is used to acquire the slope in the dynamic range control parameter and the starting point of the dynamic range in the audio metadata if the length of the media data to be processed is greater than a preset length.

[0168] The loudness estimation subunit is used to perform loudness estimation based on the target loudness, the slope and the starting point to determine the estimated loudness.

[0169] The first loudness compensation value estimation and determination subunit is configured to determine a first loudness compensation value based on a difference between the target loudness and the estimated loudness.

[0170] In some optional implementations, the first loudness compensation value determining unit further includes:

[0171] The maximum short-term loudness acquisition subunit is used to acquire the maximum short-term loudness in the audio metadata if the length of the media data to be processed is less than or equal to the preset length.

[0172] The first loudness compensation value difference determining subunit is used to determine the first loudness compensation value based on the difference between the maximum short-term loudness and the target loudness.

[0173] In some optional implementations, the performance requirement includes a first level and a second level of loudness equalization effect, the first level is lower than the second level, and if the target performance requirement includes the second level of loudness equalization effect, the first loudness compensation value determination unit includes:

[0174] The first loudness compensation model determining subunit is configured to determine a first loudness compensation value based on a first loudness compensation model for dynamic range control and audio metadata if the length of the media data to be processed is greater than a preset length.

[0175] In some optional implementations, the first loudness compensation value determining unit further includes:

[0176] The second loudness compensation model determining subunit is configured to determine a first loudness compensation value based on a second loudness compensation model for dynamic range control and audio metadata if the length of the media data to be processed is less than or equal to a preset length.

[0177] In some optional implementations, the target performance requirement is determined based on a loudness processing mode of a current playback device or an error range of loudness compensation.

[0178] In some optional implementations, the acquisition module 601 includes:

[0179] A frequency spectrum curve acquisition unit, used to acquire a frequency response curve of a target playback device, the frequency response curve is used to determine a target loudness; and / or,

[0180] The cutoff frequency acquisition unit is used to acquire the cutoff frequency of the target playback device, the cutoff frequency is used to determine the filtering loudness compensation value corresponding to the signal energy of the data to be filtered in the media data to be processed, the data to be filtered is the media data with a frequency lower than the cutoff frequency in the media data to be processed, the target playback device is used to perform loudness processing on the media data to be processed based on the filtering loudness compensation value after filtering the data to be filtered from the media data to be processed, and the audio metadata includes the target loudness and the filtering loudness compensation value.

[0181] The audio processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0182] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0183] The present disclosure also provides an electronic device having the above Figure 6 The audio processing device shown.

[0184] See also Figure 7 , Figure 7 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present disclosure, such as Figure 7As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0185] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0186] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0187] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0188] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0189] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0190] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0191] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An audio processing method, characterized in that: The method comprises: Acquire media data to be processed and audio metadata of the media data to be processed; Performing first loudness compensation on the to-be-processed media data based on the audio metadata to obtain first media data, where the first loudness compensation includes loudness compensation of dynamic range control; determining a second loudness compensation value corresponding to the peak limit based on the audio characteristics of the first media data; The first media data is subjected to second loudness compensation and peak limiting based on the second loudness compensation value to determine target media data for playback.

2. The method according to claim 1, characterized in that The determining, based on the audio feature of the first media data, a second loudness compensation value corresponding to the peak limit comprises: Obtain target performance requirements; Determining a method for determining a second loudness compensation value corresponding to the target performance requirement based on a correspondence between the performance requirement and a method for determining a loudness compensation value corresponding to the peak limit; The second loudness compensation value is determined based on a determination method of the second loudness compensation value and audio features of the first media data.

3. The method according to claim 2, characterized in that The loudness equalization effect in the performance requirement is positively correlated with the processing complexity of the method for determining the loudness compensation value of the peak limit, and the processing performance of the loudness equalization in the performance requirement is negatively correlated with the processing complexity of the method for determining the loudness compensation value of the peak limit.

4. The method according to claim 3, characterized in that The peak loudness compensation value determination method includes a determination method based on a mapping relationship between a peak loudness compensation value and an audio feature and a determination method based on a peak loudness compensation model, wherein an input of the peak loudness compensation model includes an audio feature, and an output includes the loudness compensation value; wherein a processing complexity of the determination method based on the mapping relationship between the peak loudness compensation value and the audio feature is less than a processing complexity of the determination method based on the peak loudness compensation model.

5. The method according to claim 4, characterized in that If the peak loudness compensation value determination method is a determination method based on a mapping relationship between a peak loudness compensation value and an audio feature, determining the second loudness compensation value based on the peak loudness compensation value determination method and the audio feature of the first media data includes: Acquire a target peak value of the peak limit and a current audio peak value in the audio feature, and obtain a difference between the current audio peak value and the target peak value; The second loudness compensation value is determined based on the difference and the mapping relationship.

6. The method according to claim 4, characterized in that If the peak loudness compensation value determination method is a determination method based on a peak loudness compensation model, determining the second loudness compensation value based on the peak loudness compensation value determination method and the audio feature of the first media data includes: Based on the loudness equalization effect in the target performance requirement, determining a target sub-model in the peak loudness compensation model, wherein the peak loudness compensation model includes a plurality of sub-models, and the processing complexity of each sub-model is different; The second loudness compensation value is determined based on the target sub-model and the audio features of the first media data.

7. The method according to claim 1, characterized in that The performing first loudness compensation on the to-be-processed media data based on the audio metadata to obtain first media data includes: Obtain target performance requirements; Determining a target loudness compensation value determination method for dynamic range control corresponding to the target performance requirement based on a correspondence between the performance requirement and the loudness compensation value determination method for dynamic range control; Determine a first loudness compensation value for dynamic range control based on the target loudness compensation value determination method and the audio metadata; The first loudness compensation is performed on the to-be-processed media data based on the first loudness compensation value to obtain the first media data.

8. The method according to claim 7, characterized in that The loudness equalization effect in the performance requirement is positively correlated with the processing complexity of the loudness compensation value determination method of the dynamic range control, and the processing performance of the loudness equalization in the performance requirement is negatively correlated with the processing complexity of the loudness compensation value determination method of the dynamic range control.

9. The method according to claim 8, characterized in that The performance requirement includes a first level and a second level of loudness equalization effect, the first level is lower than the second level, and if the target performance requirement includes the first level of loudness equalization effect, then determining a first loudness compensation value for dynamic range control based on the target loudness compensation value determination method and the audio metadata includes: If the length of the media data to be processed is greater than a preset length, obtaining a slope in a dynamic range control parameter and a starting point of a dynamic range in the audio metadata; Perform loudness estimation based on the target loudness, the slope, and the starting point to determine an estimated loudness; The first loudness compensation value is determined based on a difference between the target loudness and the estimated loudness.

10. The method according to claim 9, characterized in that The determining, based on the target loudness compensation value determination method and the audio metadata, a first loudness compensation value for dynamic range control further includes: If the length of the to-be-processed media data is less than or equal to the preset length, obtaining the maximum short-time loudness in the audio metadata; The first loudness compensation value is determined based on a difference between the maximum short-term loudness and the target loudness.

11. The method according to claim 8, characterized in that The performance requirement includes a first level and a second level of loudness equalization effect, the first level is lower than the second level, and if the target performance requirement includes the second level of loudness equalization effect, then determining a first loudness compensation value for dynamic range control based on the target loudness compensation value determination method and the audio metadata includes: If the length of the to-be-processed media data is greater than a preset length, the first loudness compensation value is determined based on a first loudness compensation model for dynamic range control and the audio metadata.

12. The method according to claim 11, characterized in that The determining, based on the target loudness compensation value determination method and the audio metadata, a first loudness compensation value for dynamic range control further includes: If the length of the to-be-processed media data is less than or equal to the preset length, the first loudness compensation value is determined based on a second loudness compensation model for dynamic range control and the audio metadata.

13. The method according to claim 2 or 7, characterized in that: The target performance requirement is determined based on the loudness processing mode of the current playback device or the error range of loudness compensation.

14. The method according to claim 1, characterized in that The obtaining of audio metadata of the media data to be processed comprises: Obtaining a frequency response curve of a target playback device, wherein the frequency response curve is used to determine a target loudness; and / or, A cutoff frequency of a target playback device is obtained, where the cutoff frequency is used to determine a filtering loudness compensation value corresponding to signal energy of data to be filtered in the media data to be processed, where the data to be filtered is media data in the media data to be processed whose frequency is lower than the cutoff frequency, and the target playback device is used to perform loudness processing on the media data to be processed based on the filtering loudness compensation value after filtering the data to be filtered from the media data to be processed, and the audio metadata includes the target loudness and the filtering loudness compensation value.

15. An audio processing device, characterized in that: The device comprises: An acquisition module, used to acquire the media data to be processed and the audio metadata of the media data to be processed; A first loudness processing module, configured to perform first loudness compensation on the to-be-processed media data based on the audio metadata to obtain first media data, wherein the first loudness compensation includes loudness compensation of dynamic range control; A second loudness compensation value determining module, configured to determine a second loudness compensation value corresponding to a peak value limitation based on an audio feature of the first media data; The second loudness processing module is configured to perform second loudness compensation and peak limiting on the first media data based on the second loudness compensation value, and determine target media data for playback.

16. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the audio processing method according to any one of claims 1 to 14 by executing the computer instructions.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the audio processing method according to any one of claims 1 to 14.

Images