Audio processing method and apparatus, and device and storage medium

Abstract

The present disclosure relates to the technical field of computer processing. Disclosed are an audio processing method and apparatus, and a device and a storage medium. The method provided in the present disclosure comprises: extracting audio features of media data to be published; determining a loudness equalization mode and a loudness equalization parameter of said media data on the basis of a magnitude relationship between the audio features and corresponding threshold values; and performing loudness equalization processing on audio of said media data on the basis of the loudness equalization mode and the loudness equalization parameter, and determining to publish the media data.

Description

Audio processing method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 10, 2023, with application number 202311007825.1 and invention name “Audio processing method, device, equipment and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of computer processing, and in particular to an audio processing method, apparatus, device, and storage medium. Background Art

[0003] Audio loudness equalization methods include full loudness equalization and partial loudness equalization. Full loudness equalization adjusts the audio overall based on the difference between the current loudness and the target loudness. Partial loudness equalization adjusts the audio peak to a specific level and then increases or decreases other audio levels accordingly. However, each equalization method has its pros and cons, making it difficult to choose between them in complex media data publishing scenarios.

[0004] Summary of the Invention

[0005] In view of this, the present disclosure provides an audio processing method, apparatus, device, and storage medium to solve the problem of balancing the published loudness of media data.

[0006] In a first aspect, the present disclosure provides an audio processing method, the method comprising:

[0007] Extracting audio features of the media data to be published;

[0008] Determining a loudness equalization method and loudness equalization parameters for the media data to be published based on a magnitude relationship between the audio feature and the corresponding threshold;

[0009] Based on the loudness equalization mode and the loudness equalization parameters, loudness equalization processing is performed on the audio of the to-be-published media data to determine the media data to be published.

[0010] In a second aspect, the present disclosure provides an audio processing device, the device comprising:

[0011] An audio feature extraction module, used to extract audio features of the media data to be published;

[0012] a loudness equalization determination module, configured to determine a loudness equalization method and loudness equalization parameters for the media data to be published based on a magnitude relationship between the audio feature and a corresponding threshold;

[0013] The loudness equalization processing module is used to perform loudness equalization processing on the audio of the media data to be published based on the loudness equalization method and the loudness equalization parameters, and determine the published media data.

[0014] In a third aspect, the present disclosure provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the audio processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the audio processing method of the first aspect or any corresponding embodiment thereof.

[0016] The audio processing method provided by the embodiments of the present disclosure extracts audio features of the media data to be published, compares the audio features with corresponding thresholds, and determines the loudness equalization method and equalization parameters for the media data to be published. Specifically, the determined loudness equalization method and equalization parameters are based on and closely related to the media data to be published. Different media data to be published utilize their respective audio features to obtain corresponding loudness equalization methods and equalization parameters, thereby enabling adaptive loudness equalization processing of the media data to be published. This method is universally applicable in most scenarios, can meet publishing requirements to the greatest extent possible, properly ensure that audio quality is not affected, and ensure that good sound quality can be reproduced on most devices when the consumer plays back the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] FIG1 is a processing block diagram of an audio processing method according to an embodiment of the present disclosure;

[0019] FIG2 is a flow chart of an audio processing method according to an embodiment of the present disclosure;

[0020] FIG3 is a flow chart of another audio processing method according to an embodiment of the present disclosure;

[0021] FIG4 is a flow chart of another audio processing method according to an embodiment of the present disclosure;

[0022] FIG5 is a structural block diagram of an audio processing device according to an embodiment of the present disclosure;

[0023] FIG6 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present disclosure.

[0025] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0026] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0027] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0029] In related art, loudness equalization is performed based on user selection. Specifically, when a user publishes media data, a computer device may provide an interface or other means for the user to select a loudness equalization method. Based on experience, the user interacts with the computer device and selects either full loudness equalization or partial loudness equalization to perform loudness equalization on the media data, which is then processed and published.

[0030] For example, there are two ways to normalize the published loudness of media data: upward normalization and downward normalization. When the published loudness setting (e.g., -18) is lower than the loudness of the original media data (e.g., -13), a downward gain of 5dB is applied. Similarly, when the published loudness setting is higher than the loudness of the original video (e.g., -22), an upward gain of 4dB is required. However, the amount of upward gain depends on whether the algorithm uses partial or full loudness equalization. Partial loudness equalization sets an upper limit for the peak amplitude of the audio, for example, a peak of -1dB. If a video with a loudness of -22 has a peak amplitude of -3dBfs, then an upward gain of 4dB will bring the peak to 1dBfs. In this case, partial loudness equalization will only result in a 2dB gain, while full loudness equalization is not limited by the audio amplitude and can still achieve a 4dB gain.

[0031] Among them, the loudness equalization method used for the media data to be published is based on the user's choice, which may cause the sound quality of the published media data obtained after loudness equalization to be greatly affected. Based on this, the audio processing method provided by the embodiment of the present disclosure extracts the audio features of the media data to be published, and determines the loudness equalization method and loudness equalization parameters on this basis. On this basis, the audio of the media data to be published is subjected to loudness equalization processing, the published media data is determined, and then it is published. This method can meet the publishing requirements to the greatest extent, properly ensure that the audio quality is not affected, and ensure that good sound quality can be reproduced on most devices when the consumer side is played back. For example, Figure 1 shows a module schematic diagram of the audio processing method, after mixing and other processing of the audio data of audio track 1 to audio track N, the media data to be published is obtained; the audio features of the media data to be published are extracted and loudness equalized to obtain the published media data, and finally the published media data is published.

[0032] It should be noted that the audio processing methods provided in the embodiments of this disclosure are applied to loudness equalization processing on the media data creation side. For example, when a user uploads media data to be published through application software on a computer device, the audio processing methods provided in the embodiments of this disclosure can be used to perform loudness equalization processing on the media data before publishing it.

[0033] According to an embodiment of the present disclosure, an embodiment of an audio processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] In this embodiment, an audio processing method is provided, which can be used in computer devices, such as mobile terminals and computers. FIG2 is a flow chart of the audio processing method according to an embodiment of the present disclosure. As shown in FIG2 , the flow chart includes the following steps:

[0035] Step S201: extracting audio features of the media data to be published.

[0036] The media data to be published includes, but is not limited to, audio data or video data to be published. The specific form of the media data is not limited and is determined based on actual needs. The media data to be published can be obtained by a user through filming or recording, and then uploaded to a computer device. Accordingly, the media data to be published can be obtained.

[0037] For example, if the computer device is a mobile phone, after the user determines the media data to be published, the user publishes the media data to be published through the application software on the mobile phone. Accordingly, the media data to be published can be obtained when the user authorizes the user.

[0038] The audio features of the media data to be published include, but are not limited to, one or more of the following: current loudness, energy, spectrum, phase, dynamic range, and voice percentage. The specific audio features to be extracted are determined based on user needs. Audio features can be extracted using a neural network-based feature extraction module or by performing waveform or spectrum analysis on the audio data in the media data to be published. The specific method for extracting audio features is not limited here and can be set based on actual circumstances.

[0039] Step S202 : determining a loudness equalization method and loudness equalization parameters for the media data to be published based on a magnitude relationship between the audio feature and the corresponding threshold.

[0040] It should be noted that there is a one-to-one correspondence between audio features and thresholds, i.e., different audio features correspond to different thresholds. The thresholds corresponding to each audio feature are set based on actual needs, and their specific values ​​are not limited here.

[0041] For some audio features, if the audio feature is greater than a corresponding threshold, the loudness equalization method for the media data to be published is determined to be full loudness equalization, and corresponding loudness equalization parameters are obtained accordingly using the audio feature. If the audio feature is less than or equal to the corresponding threshold, the loudness equalization method for the media data to be published is determined to be partial loudness equalization, and corresponding loudness equalization parameters are obtained accordingly using the audio feature. The loudness equalization parameters for full loudness equalization include gain, and the loudness equalization parameters for partial loudness equalization include gain and peak.

[0042] For other audio features, if the audio feature is greater than the corresponding threshold, the loudness equalization mode of the media data to be published is determined to be partial loudness equalization; otherwise, the loudness equalization mode of the media data to be published is determined to be full loudness equalization.

[0043] Therefore, the relationship between the audio features and the corresponding thresholds, and the corresponding loudness equalization method, is related to the specific audio features. Based on this, the relationship between the audio features and the corresponding thresholds and the corresponding loudness equalization method are stored, and the corresponding loudness equalization method can be determined based on the comparison results of the specific relationship.

[0044] Step S203 : performing loudness equalization processing on the audio of the media data to be published based on the loudness equalization mode and the loudness equalization parameters, and determining to publish the media data.

[0045] After the loudness equalization method is determined, the audio of the to-be-published media data is subjected to loudness equalization processing using the corresponding loudness equalization method and loudness equalization parameters to obtain the published media data, which is then published.

[0046] The audio processing method provided in this embodiment extracts audio features of the media data to be published, compares the audio features with corresponding thresholds, and determines the loudness equalization method and equalization parameters for the media data to be published. Specifically, the determined loudness equalization method and equalization parameters are based on and closely related to the media data to be published. Different media data to be published utilize their respective audio features to obtain corresponding loudness equalization methods and equalization parameters, thereby enabling adaptive loudness equalization processing of the media data to be published. This method is universally applicable in most scenarios, can meet publishing requirements to the greatest extent possible, properly ensures that audio quality is not affected, and guarantees that good sound quality can be reproduced on most devices when the consumer plays back the audio.

[0047] In this embodiment, an audio processing method is provided, which can be used in computer devices, such as mobile terminals and computers. FIG3 is a flow chart of the audio processing method according to an embodiment of the present disclosure. As shown in FIG3 , the flow chart includes the following steps:

[0048] Step S301: extracting audio features of the media data to be published. Please refer to step S301 of the embodiment shown in FIG2 for details, which will not be repeated here.

[0049] Step S302 : determining a loudness equalization method and loudness equalization parameters for the media data to be published based on a magnitude relationship between the audio feature and the corresponding threshold.

[0050] Specifically, the above step S302 includes:

[0051] Step S3032: Determine a loudness equalization method for the media data to be published based on a relationship between the audio feature and the corresponding threshold.

[0052] In step S202 of the embodiment shown in FIG. 2 , the relationship between the audio features and the corresponding thresholds and the specific implementation of determining the loudness equalization method are described, which will not be repeated here.

[0053] In some optional implementations, the above step S3021 includes: if the audio feature includes a dynamic range, determining a loudness equalization method based on a relationship between the dynamic range and a dynamic range threshold.

[0054] Dynamic range is the relative value between the maximum and minimum sound pressure levels. For example, if the maximum sound pressure level of a piece of music is 90dB and the minimum sound pressure level is 30dB, then the dynamic range of this piece of music is 60dB. After processing in step S302 above, the dynamic range of the media data to be published is obtained. The dynamic range is compared with the dynamic range threshold. If the dynamic range is greater than the dynamic range threshold, the loudness equalization method is determined to be full loudness equalization; otherwise, the loudness equalization method is determined to be partial loudness equalization.

[0055] For example, the dynamic range threshold is set to 24. If the dynamic range is greater than 24, the loudness equalization mode is determined to be full loudness equalization; if the dynamic range is less than or equal to 24, the loudness equalization mode is determined to be partial loudness equalization.

[0056] In some optional implementations, step S3031 includes: if the audio features include loudness sampling points, counting the proportion of loudness sampling points whose loudness exceeds a preset peak value, and determining a loudness equalization method based on a relationship between the proportion and a proportion threshold.

[0057] The audio features include loudness sampling points. The loudness value of each loudness sampling point is compared with a preset peak value. The number of loudness sampling points exceeding the preset peak value is counted. The ratio of this counted number to the total number of loudness sampling points is calculated to determine the percentage of loudness sampling points exceeding the preset peak value. The calculated percentage is then compared with a percentage threshold. If the percentage is less than the percentage threshold, the loudness equalization method is determined to be full loudness equalization; otherwise, the loudness equalization method is determined to be partial loudness equalization.

[0058] For example, if the preset peak value is 0dbfs, if the proportion of loudness sampling points exceeding 0dbfs is less than 5%, the loudness equalization mode is determined to be full loudness equalization; if the proportion of loudness sampling points exceeding 0dbfs is greater than or equal to 5%, the loudness equalization mode is determined to be partial loudness equalization.

[0059] Since the dynamic range and the loudness sampling points can be directly obtained from the audio features, using the dynamic range or the loudness sampling points to determine the loudness equalization method can improve the efficiency of determining the loudness equalization method.

[0060] It should be noted that the above are merely two optional methods, namely, determining the loudness equalization method by using the dynamic range or the proportion of loudness sampling points exceeding a preset peak value. However, the scope of protection of the present disclosure is not limited thereto. The specific audio features adopted are set according to actual needs and are not limited thereto herein.

[0061] Step S3022: If the loudness equalization mode is complete loudness equalization, the target loudness of the to-be-published media data is determined based on the audio features.

[0062] For complete loudness equalization, the corresponding loudness equalization parameters include gain, which is determined by the target loudness of the media data to be published and the current loudness. The current loudness is obtained by extracting the audio features in step S301 above. Therefore, it is necessary to determine the target loudness of the media data to be published based on the audio features.

[0063] In some optional implementations, step S3022 includes:

[0064] Step a1: Obtain the preset release loudness.

[0065] Step a2: determining a target loudness based on the audio features and a preset release loudness.

[0066] The preset release loudness is set based on actual needs, and its specific value is not limited here. The target loudness is calculated based on the preset release loudness and combined with audio characteristics. For example, the target loudness can be calculated based on the speech ratio or by offsetting the preset release loudness.

[0067] The target loudness is obtained by using a preset release loudness and audio features. Since the preset release loudness can guarantee release expectations, combining the preset release loudness with the audio features can further improve the reliability of the obtained target loudness.

[0068] In some optional implementations, the audio feature includes a voice ratio. Accordingly, step a2 includes:

[0069] Step a211: Determine a first loudness adjustment value based on the voice ratio.

[0070] Step a212: Adjust the preset published loudness based on the first loudness adjustment value to determine a target loudness.

[0071] The voice proportion is obtained by extracting audio features of the media data to be published, converting the voice proportion into a first loudness adjustment value according to a ratio, and then adjusting the preset publishing loudness according to the first loudness adjustment value to obtain the target loudness.

[0072] For example, the target loudness is determined using the following formula:

[0073] Here, X is the preset loudness, for example, X=-16.

[0074] Because users perceive the loudness of speech and music differently, the first loudness adjustment value is determined based on the speech ratio, and the target loudness is obtained based on this value. This further improves the reliability of the target loudness and ensures the sound quality of the published media data obtained after subsequent loudness equalization processing.

[0075] In some optional implementations, the audio feature includes the type of audio content. Accordingly, step a2 includes:

[0076] Step a221: Obtain a second loudness adjustment value corresponding to the type of audio content.

[0077] Step a222: Adjust the preset published loudness based on the second loudness adjustment value to determine a target loudness.

[0078] Different audio content types have different loudness ranges. Accordingly, the human ear perceives audio loudness ranges differently. Therefore, the preset publishing loudness can be adjusted based on the type of audio content. For example, if the audio content is music, the corresponding second loudness adjustment value is smaller, while if the audio content is speech, the corresponding second loudness adjustment value is larger. Based on this, a correspondence between different audio content types and second loudness adjustment values ​​is established. After determining the type of audio content in the media data to be published, this correspondence can be used to determine the corresponding second loudness adjustment value.

[0079] After the second loudness adjustment value is determined, the target loudness can be determined by increasing or decreasing the corresponding second loudness adjustment value based on the preset published loudness.

[0080] The type of the audio content can ensure the type of the media data to be published, and can be used to determine the corresponding second loudness adjustment, thereby further improving the reliability of the target loudness.

[0081] Step S3023: Determine loudness equalization parameters corresponding to complete loudness equalization based on the target loudness and the current loudness in the audio features.

[0082] As described above, the loudness equalization parameter corresponding to the complete loudness equalization includes gain. Based on this, in the loudness equalization mode of complete loudness equalization, gain=the difference between the target loudness and the current loudness.

[0083] In some optional implementations, step S3023 includes: calculating a difference between the target loudness and the current loudness, and determining the difference as a first gain corresponding to complete loudness equalization, where the loudness equalization parameter includes the first gain.

[0084] To distinguish it from the gain corresponding to the subsequent partial loudness equalization, the gain corresponding to the full loudness equalization is referred to as the first gain. For example, if the target loudness is -18 and the current loudness is -22, then the first gain = -18 - (-22) = 4.

[0085] On the basis of the obtained target loudness with higher reliability, the first gain is obtained by utilizing the difference between the target loudness and the current loudness, thereby ensuring the accuracy of the first gain.

[0086] Step S3024: If the loudness equalization mode is partial loudness equalization, a second gain corresponding to the partial loudness equalization is determined based on the current peak value and the preset peak value in the audio feature.

[0087] The loudness equalization parameters include a preset peak value and a second gain.

[0088] The second gain may be the difference between the preset peak value and the current peak value. For example, if the preset peak value is 0 dbfs and the current peak value is -1 dbfs, the second gain = 0 - (-1) = 1. In partial loudness equalization, the loudness equalization parameters include the preset peak value and the second gain.

[0089] Step S303: Based on the loudness equalization method and loudness equalization parameters, the audio of the media data to be published is subjected to loudness equalization processing to determine the published media data. For details, please refer to the detailed description of step S203 in the embodiment shown in FIG2 , which will not be repeated here.

[0090] The audio processing method provided in this embodiment, when full loudness equalization is used, determines the target loudness of the media data to be published based on audio features. This is then combined with the current loudness to determine the loudness equalization parameters. Because the loudness equalization parameters are derived based on the audio features, the reliability of the resulting loudness equalization parameters is improved, thereby ensuring the sound quality of the resulting media data. When partial loudness equalization is used, the second gain is determined using the current peak value and a preset peak value. This and the preset peak value are used as the loudness equalization parameters, ensuring the accuracy of the resulting loudness equalization parameters.

[0091] As a specific application example of the disclosed embodiments, after a user creates a short video to be published, they publish it through a mobile phone application. After the user uploads the short video to be published through the application, the application, upon obtaining user authorization, performs loudness equalization processing on the short video to be published. For example, as shown in FIG4 , the audio features of the short video to be published are extracted, and the dynamic range corresponding to the short video to be published is compared with a dynamic range threshold of 24, or the proportion of sampling points exceeding 0dBFS in the short video to be published is compared with a proportion threshold of 5%. If the dynamic range is greater than 24, or the proportion of sampling points exceeding 0dBFS is less than 5%, the loudness equalization method is determined to be full loudness equalization; otherwise, the loudness equalization method is determined to be partial loudness equalization. If the loudness equalization method is full loudness equalization, a target loudness (e.g., -18) is determined based on the type of audio content or the proportion of speech, and then a first gain (gain = -18 - (-22) = 4) is calculated based on the target loudness. If the loudness equalization method is partial loudness equalization, a preset peak value (e.g., 0 dBfs) is obtained and a second gain is calculated: gain = 0 - (-1) = 1. After determining the loudness equalization method and loudness equalization parameters, the short video to be published is subjected to loudness equalization processing to obtain the published short video, which can then be published.

[0092] This embodiment also provides an audio processing device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0093] This embodiment provides an audio processing device, as shown in FIG5 , including:

[0094] The audio feature extraction module 501 is used to extract audio features of the media data to be published.

[0095] The loudness equalization determination module 502 is configured to determine a loudness equalization method and loudness equalization parameters for the media data to be published based on a magnitude relationship between audio features and corresponding thresholds.

[0096] The loudness equalization processing module 503 is configured to perform loudness equalization processing on the audio of the media data to be published based on the loudness equalization method and loudness equalization parameters, and determine to publish the media data.

[0097] In some optional implementations, the loudness equalization parameter determination method in the loudness equalization determination module 502 includes:

[0098] The target loudness determination unit is configured to determine the target loudness of the to-be-published media data based on the audio features if the loudness equalization mode is complete loudness equalization.

[0099] The loudness equalization determination unit is configured to determine loudness equalization parameters corresponding to complete loudness equalization based on the target loudness and the current loudness in the audio feature.

[0100] In some optional implementations, the target loudness determination unit includes:

[0101] The loudness acquisition subunit is used to obtain the preset release loudness.

[0102] The target loudness determination subunit is configured to determine the target loudness based on the audio characteristics and the preset release loudness.

[0103] In some optional implementations, the audio feature includes a speech ratio, and the target loudness determination subunit includes:

[0104] The first loudness adjustment value determining subunit is configured to determine a first loudness adjustment value based on a speech ratio.

[0105] The first adjustment subunit is configured to adjust the preset published loudness based on the first loudness adjustment value to determine the target loudness.

[0106] In some optional implementations, the audio feature includes the type of audio content, and the target loudness determination subunit includes:

[0107] The second loudness adjustment value obtaining subunit is configured to obtain a second loudness adjustment value corresponding to the type of audio content.

[0108] The second adjustment subunit is configured to adjust the preset published loudness based on the second loudness adjustment value to determine the target loudness.

[0109] In some optional implementations, the loudness equalization determination unit includes:

[0110] The calculation subunit is configured to calculate a difference between the target loudness and the current loudness, and determine the difference as a first gain corresponding to complete loudness equalization, where the loudness equalization parameter includes the first gain.

[0111] In some optional implementations, the loudness equalization parameter determination method in the loudness equalization determination module 502 includes:

[0112] The partial loudness equalization unit is configured to determine, if the loudness equalization mode is partial loudness equalization, a second gain corresponding to the partial loudness equalization based on a current peak value and a preset peak value in the audio feature, where the loudness equalization parameters include the preset peak value and the second gain.

[0113] In some optional implementations, the loudness equalization determination module 502 includes:

[0114] The first loudness equalization mode determining unit is configured to determine a loudness equalization mode based on a magnitude relationship between the dynamic range and a dynamic range threshold if the audio feature includes a dynamic range.

[0115] The second loudness equalization mode determining unit is configured to, if the audio feature includes loudness sampling points, count the proportion of loudness sampling points whose loudness exceeds a preset peak value, and determine the loudness equalization mode based on a relationship between the proportion and a proportion threshold.

[0116] The audio processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0117] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0118] The embodiment of the present disclosure further provides a computer device having the audio processing device shown in FIG. 5 .

[0119] Please refer to Figure 6, which is a structural diagram of a computer device provided by an optional embodiment of the present disclosure. As shown in Figure 6, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 takes a processor 10 as an example.

[0120] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0121] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0122] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0123] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0124] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means, and FIG6 takes the bus connection as an example.

[0125] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, a pointer, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0126] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0127] Although the embodiments of the present disclosure have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An audio processing method, wherein: The method comprises: Extracting audio features of media data to be published; Determining a loudness equalization method and loudness equalization parameters of the to-be-published media data based on a magnitude relationship between the audio feature and the corresponding threshold; Based on the loudness equalization mode and the loudness equalization parameters, loudness equalization processing is performed on the audio of the media data to be published, and the media data to be published is determined.

2. The method according to claim 1, wherein: The method for determining the loudness equalization parameter includes: If the loudness equalization mode is complete loudness equalization, determining a target loudness of the to-be-published media data based on the audio feature; Based on the target loudness and the current loudness in the audio feature, a loudness equalization parameter corresponding to the complete loudness equalization is determined.

3. The method according to claim 2, wherein: The determining the target loudness of the to-be-published media data based on the audio feature includes: Get the preset release loudness; The target loudness is determined based on the audio feature and the preset release loudness.

4. The method according to claim 3, wherein: The audio feature includes a voice ratio, and determining the target loudness based on the audio feature and the preset release loudness includes: Determining a first loudness adjustment value based on the voice proportion; The preset published loudness is adjusted based on the first loudness adjustment value to determine the target loudness.

5. The method according to claim 3, wherein: The audio feature includes a type of audio content, and determining the target loudness based on the audio feature and the preset release loudness includes: Obtaining a second loudness adjustment value corresponding to the type of the audio content; The preset release loudness is adjusted based on the second loudness adjustment value to determine the target loudness. Spend.

6. The method according to claim 2, wherein: The determining, based on the target loudness and the current loudness in the audio feature, a loudness equalization parameter corresponding to the complete loudness equalization includes: A difference between the target loudness and the current loudness is calculated, and the difference is determined as a first gain corresponding to the complete loudness equalization, wherein the loudness equalization parameter includes the first gain.

7. The method according to claim 1, wherein: The method for determining the loudness equalization parameter includes: If the loudness equalization mode is partial loudness equalization, a second gain corresponding to the partial loudness equalization is determined based on a current peak value and a preset peak value in the audio feature, and the loudness equalization parameters include the preset peak value and the second gain.

8. The method according to any one of claims 1 to 7, wherein: The determining, based on the magnitude relationship between the audio feature and the corresponding threshold, a loudness balancing method of the to-be-published media data includes: If the audio feature includes a dynamic range, determining the loudness equalization method based on a magnitude relationship between the dynamic range and a dynamic range threshold; If the audio feature includes loudness sampling points, the proportion of loudness sampling points whose loudness exceeds a preset peak value is counted, and the loudness equalization method is determined based on a magnitude relationship between the proportion and a proportion threshold.

9. An audio processing device, wherein: The device comprises: An audio feature extraction module, used to extract audio features of the media data to be published; A loudness equalization determination module, configured to determine a loudness equalization method and loudness equalization parameters of the to-be-published media data based on a magnitude relationship between the audio feature and a corresponding threshold; The loudness equalization processing module is used to perform loudness equalization processing on the audio of the media data to be published based on the loudness equalization method and the loudness equalization parameters, and determine to publish the media data.

10. A computer device, wherein: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the audio processing method according to any one of claims 1 to 8 by executing the computer instructions.

11. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the audio processing method according to any one of claims 1 to 8.

Images