Audio processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of computer processing, and discloses an audio processing method and device, equipment and a storage medium, and the method comprises the steps: extracting audio features of to-be-published media data; determining a loudness equalization mode and loudness equalization parameters of the to-be-published media data based on a size relationship between the audio features and corresponding thresholds; and based on the loudness equalization mode and the loudness equalization parameter, carrying out loudness equalization processing on the audio of the to-be-published media data, and determining published media data.

Description

Technical Field

[0001] The present disclosure relates to the field of computer processing, and in particular to an audio processing method, apparatus, device and storage medium. Background Art

[0002] There are two ways to equalize audio loudness: full loudness equalization and partial loudness equalization. Full loudness equalization is to adjust the audio as a whole based on the difference between the current loudness and the target loudness; partial loudness equalization is to adjust the audio peak to a specific size, and then make corresponding increases or decreases in other audio. However, these two equalization methods have their own advantages and disadvantages. In complex media data publishing scenarios, it is difficult to choose between the two methods. Summary of the invention

[0003] In view of this, the present disclosure provides an audio processing method, apparatus, device and storage medium to solve the problem of balancing the publishing loudness of media data.

[0004] In a first aspect, the present disclosure provides an audio processing method, the method comprising:

[0005] Extracting audio features of media data to be published;

[0006] Determining a loudness equalization method and loudness equalization parameters of the to-be-published media data based on a magnitude relationship between the audio feature and the corresponding threshold;

[0007] Based on the loudness equalization mode and the loudness equalization parameters, loudness equalization processing is performed on the audio of the media data to be published, and the media data to be published is determined.

[0008] In a second aspect, the present disclosure provides an audio processing device, the device comprising:

[0009] An audio feature extraction module, used to extract audio features of the media data to be published;

[0010] A loudness equalization determination module, configured to determine a loudness equalization method and loudness equalization parameters of the to-be-published media data based on a magnitude relationship between the audio feature and a corresponding threshold;

[0011] The loudness equalization processing module is used to perform loudness equalization processing on the audio of the media data to be published based on the loudness equalization method and the loudness equalization parameters, and determine to publish the media data.

[0012] In a third aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the audio processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0013] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the audio processing method of the first aspect or any corresponding embodiment thereof.

[0014] The audio processing method provided by the embodiment of the present disclosure extracts the audio features of the media data to be published, compares the audio features with the corresponding thresholds, and determines the loudness equalization mode and equalization parameters of the media data to be published. That is, the determined loudness equalization mode and equalization parameters are determined based on the media data to be published itself and are closely related to the media data to be published. Different media data to be published use their own audio features to obtain corresponding loudness equalization modes and equalization parameters, so as to perform adaptive loudness equalization processing on the media data to be published. The method can be universally applied in most scenarios, and can meet the publishing requirements to the greatest extent, properly ensure that the audio quality is not affected, and ensure that good sound quality can be reproduced on most devices when the consumer side plays back. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 is a processing block diagram of an audio processing method according to an embodiment of the present disclosure;

[0017] Figure 2 is a flowchart of an audio processing method according to an embodiment of the present disclosure;

[0018] Figure 3 is a flowchart of another audio processing method according to an embodiment of the present disclosure;

[0019] Figure 4 is a flowchart of another audio processing method according to an embodiment of the present disclosure;

[0020] Figure 5 is a structural block diagram of an audio processing device according to an embodiment of the present disclosure;

[0021] Figure 6 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0023] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0024] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0025] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0026] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0027] In the related art, the loudness equalization processing is performed according to the user's selection, that is, when the user publishes the media data, the computer device can provide an interface for the user to select the loudness equalization method. The user selects one of the media data, complete loudness equalization or partial loudness equalization, through interaction with the computer device based on experience, and then obtains the published media data after the loudness equalization processing, and then publishes it.

[0028] For example, there are two ways to normalize the published loudness of media data: upward normalization and downward normalization. When the published loudness setting (e.g., -18) is lower than the loudness of the original media data (e.g., -13), a downward gain of 5db is applied. Similarly, when the published loudness setting is higher than the loudness of the original video (e.g., -22), an upward gain of 4db is required, but the upward gain depends on whether the loudness equalization method used by the algorithm is partial loudness equalization or full loudness equalization. Partial loudness equalization sets an upper limit for the peak amplitude of the audio, for example, a peak of -1db. If a video with a loudness of -22 is assumed to have a peak amplitude of -3dbfs, then an upward gain of 4db is applied, and the peak reaches 1dbfs. At this time, partial loudness equalization will actually only gain 2db, while full loudness equalization is not limited by the audio amplitude and can still gain 4db.

[0029] Among them, the loudness equalization method used for the media data to be published is based on the user's choice, which may cause the sound quality of the published media data obtained after loudness equalization to be greatly affected. Based on this, the audio processing method provided by the embodiment of the present disclosure extracts the audio features of the media data to be published, and determines the loudness equalization method and loudness equalization parameters on this basis. On this basis, the audio of the media data to be published is processed with loudness equalization, the media data to be published is determined, and then it is published. This method can meet the publishing requirements to the greatest extent, properly ensure that the audio quality is not affected, and ensure that good sound quality can be reproduced on most devices when the consumer side plays back. For example, Figure 1 A module schematic diagram of an audio processing method is shown. After mixing and other processing are performed on the audio data of tracks 1 to N, media data to be published is obtained; audio features of the media data to be published are extracted for loudness equalization to obtain published media data, and finally the published media data is published.

[0030] It should be noted that the audio processing method provided in the embodiment of the present disclosure is a loudness equalization process applied to the media data creation side. For example, when a user uploads the media data to be published through the application software in the computer device, the audio processing method provided in the embodiment of the present disclosure performs loudness equalization processing on the media data to be published before publishing.

[0031] According to an embodiment of the present disclosure, an audio processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] In this embodiment, an audio processing method is provided, which can be used in computer devices, such as mobile terminals and computers. Figure 2is a flowchart of an audio processing method according to an embodiment of the present disclosure, such as Figure 2 As shown, the process includes the following steps:

[0033] Step S201: extracting audio features of the media data to be published.

[0034] The media data to be published includes but is not limited to the audio data to be published or the video data to be published. There is no specific limitation on its form, which is determined according to actual needs. The media data to be published can be obtained by the user through shooting or recording and then uploaded to the computer device. Accordingly, the media data to be published can be obtained.

[0035] For example, if the computer device is a mobile phone, after the user determines the media data to be published, the user publishes the media data to be published through the application software on the mobile phone. Accordingly, the media data to be published can be obtained with the user's authorization.

[0036] The audio features of the media data to be published include but are not limited to one or more of the current loudness, energy, spectrum, phase, dynamic range, and voice ratio. The specific audio features to be extracted are determined according to user needs. The audio features can be extracted by a feature extraction module based on a neural network, or by analyzing the waveform or spectrum of the audio data in the media data to be published. The specific method of extracting audio features is not limited here, and it can be set according to the actual situation.

[0037] Step S202: determining a loudness equalization method and loudness equalization parameters of the media data to be published based on a magnitude relationship between the audio feature and the corresponding threshold.

[0038] It should be noted that there is a one-to-one correspondence between audio features and thresholds, that is, different audio features have different corresponding thresholds. The thresholds corresponding to each audio feature are set according to actual needs, and no limitation is made on their specific values.

[0039] For some audio features, if the audio feature is greater than the corresponding threshold, it is determined that the loudness equalization mode of the media data to be published is complete loudness equalization, and accordingly, the corresponding loudness equalization parameters are obtained using the audio feature; if the audio feature is less than or equal to the corresponding threshold, it is determined that the loudness equalization mode of the media data to be published is partial loudness equalization, and accordingly, the corresponding loudness equalization parameters are obtained using the audio feature. Among them, the loudness equalization parameters of the complete loudness equalization include gain, and the loudness equalization parameters of the partial loudness equalization include gain and peak.

[0040] For other audio features, if the audio features are greater than corresponding thresholds, the loudness equalization mode of the media data to be published is determined to be partial loudness equalization; otherwise, the loudness equalization mode of the media data to be published is determined to be full loudness equalization.

[0041] Therefore, the magnitude relationship between the audio feature and the corresponding threshold value, and the corresponding loudness equalization method, are related to the specific audio feature. Based on this, the magnitude relationship between the audio feature and the corresponding threshold value, and the corresponding loudness equalization method are stored, and the corresponding loudness equalization method can be determined based on the comparison result of the specific magnitude relationship.

[0042] Step S203: Based on the loudness equalization mode and the loudness equalization parameters, loudness equalization processing is performed on the audio of the media data to be published, and the published media data is determined.

[0043] After the loudness equalization method is determined, the audio of the to-be-published media data is processed for loudness equalization using the corresponding loudness equalization method and loudness equalization parameters to obtain the published media data, which is then published.

[0044] The audio processing method provided in this embodiment extracts audio features of the media data to be published, compares the audio features with the corresponding thresholds, and determines the loudness equalization mode and equalization parameters of the media data to be published. That is, the determined loudness equalization mode and equalization parameters are determined based on the media data to be published itself and are closely related to the media data to be published. Different media data to be published use their own audio features to obtain corresponding loudness equalization modes and equalization parameters, so as to perform adaptive loudness equalization processing on the media data to be published. This method can be universally applied in most scenarios, and can meet the publishing requirements to the greatest extent, properly ensure that the audio quality is not affected, and ensure that good sound quality can be reproduced on most devices when the consumer side plays back.

[0045] In this embodiment, an audio processing method is provided, which can be used in computer devices, such as mobile terminals and computers. Figure 3 is a flowchart of an audio processing method according to an embodiment of the present disclosure, such as Figure 3 As shown, the process includes the following steps:

[0046] Step S301: extract the audio features of the media data to be published. Figure 2 Step S301 of the illustrated embodiment will not be described in detail here.

[0047] Step S302: determining a loudness equalization method and loudness equalization parameters of the media data to be published based on a magnitude relationship between the audio feature and the corresponding threshold.

[0048] Specifically, the above step S302 includes:

[0049] Step S3032: determining a loudness balancing method for the media data to be published based on a magnitude relationship between the audio feature and the corresponding threshold.

[0050] exist Figure 2 Step S202 of the illustrated embodiment describes the relationship between the audio feature and the corresponding threshold value, and the specific implementation of determining the loudness equalization method, which will not be repeated here.

[0051] In some optional implementations, the above step S3021 includes: if the audio feature includes a dynamic range, determining a loudness equalization method based on a size relationship between the dynamic range and a dynamic range threshold.

[0052] The dynamic range is the relative value between the maximum sound pressure level and the minimum sound pressure level. For example, if the maximum sound pressure level of a piece of music is 90dB and the minimum sound pressure level is 30dB, then the dynamic range of this piece of music is 60dB. After the above step S302 is processed, the dynamic range of the media data to be published is obtained. The dynamic range is compared with the dynamic range threshold. If the dynamic range is greater than the dynamic range threshold, the loudness equalization mode is determined to be full loudness equalization; otherwise, the loudness equalization mode is determined to be partial loudness equalization.

[0053] For example, the dynamic range threshold is set to 24. If the dynamic range is greater than 24, the loudness equalization mode is determined to be full loudness equalization; if the dynamic range is less than or equal to 24, the loudness equalization mode is determined to be partial loudness equalization.

[0054] In some optional implementations, the above step S3031 includes: if the audio feature includes loudness sampling points, counting the proportion of loudness sampling points whose loudness exceeds a preset peak value, and determining the loudness equalization method based on the relationship between the proportion and the proportion threshold.

[0055] The audio features include loudness sampling points. The loudness value of each loudness sampling point is compared with a preset peak value. The number of loudness sampling points exceeding the preset peak value is counted. The ratio of the counted number to the number of all loudness sampling points is calculated to obtain the proportion of loudness sampling points exceeding the preset peak value. The calculated proportion is compared with the proportion threshold. If the proportion is less than the proportion threshold, the loudness equalization mode is determined to be full loudness equalization; otherwise, the loudness equalization mode is determined to be partial loudness equalization.

[0056] For example, the preset peak value is 0dbfs. If the proportion of loudness sampling points exceeding 0dbfs is <5%, the loudness equalization mode is determined to be full loudness equalization; if the proportion of loudness sampling points exceeding 0dbfs is >= 5%, the loudness equalization mode is determined to be partial loudness equalization.

[0057] Since the dynamic range and the loudness sampling points can be directly obtained from the audio features, using the dynamic range or the loudness sampling points to determine the loudness equalization method can improve the efficiency of determining the loudness equalization method.

[0058] It should be noted that the above are only two optional methods, that is, using the dynamic range or the proportion of loudness sampling points exceeding the preset peak value to determine the loudness equalization method, but the protection scope of the present disclosure is not limited to this. The specific audio features used are set according to actual needs and are not limited here.

[0059] Step S3022: If the loudness equalization mode is complete loudness equalization, the target loudness of the media data to be published is determined based on the audio features.

[0060] For complete loudness equalization, the corresponding loudness equalization parameters include gain, which is determined by the target loudness of the media data to be published and the current loudness. The current loudness is obtained by extracting the audio features in step S301. Therefore, it is necessary to determine the target loudness of the media data to be published based on the audio features.

[0061] In some optional implementations, step S3022 includes:

[0062] Step a1, obtaining a preset release loudness.

[0063] Step a2: determining a target loudness based on the audio features and a preset release loudness.

[0064] The preset release loudness is set according to actual needs, and its specific value is not limited here. Based on the preset release loudness, the target loudness is obtained by combining the audio features. For example, the target loudness can be calculated by voice proportion, or by offsetting the preset release loudness.

[0065] The target loudness is obtained by using the preset release loudness and audio features. Since the preset release loudness can guarantee the release expectation, the reliability of the obtained target loudness can be further improved by combining the preset release loudness with the audio features.

[0066] In some optional implementations, the audio feature includes a voice ratio, and accordingly, step a2 includes:

[0067] Step a211, determining a first loudness adjustment value based on the voice proportion.

[0068] Step a212: adjusting the preset published loudness based on the first loudness adjustment value to determine a target loudness.

[0069] The voice proportion is obtained by extracting audio features of the media data to be published, converting the voice proportion into a first loudness adjustment value according to a ratio, and then adjusting the preset publishing loudness according to the first loudness adjustment value to obtain a target loudness.

[0070] For example, the target loudness is determined using the following formula:

[0071]

[0072] Wherein, X is a preset loudness, for example, X=-16.

[0073] Since users perceive the loudness of speech and music differently, the first loudness adjustment value is determined by the speech proportion, and the target loudness is obtained on this basis, which further improves the reliability of the target loudness and ensures the sound quality of the published media data obtained after subsequent loudness equalization processing.

[0074] In some optional implementations, the audio feature includes the type of audio content. Accordingly, step a2 includes:

[0075] Step a221: Obtain a second loudness adjustment value corresponding to the type of audio content.

[0076] Step a222: adjusting the preset published loudness based on the second loudness adjustment value to determine the target loudness.

[0077] Different types of audio content have different loudness ranges. Accordingly, human ears perceive different loudness ranges of audio. Therefore, the preset publishing loudness can be adjusted in combination with the type of audio content. For example, if the type of audio content is music, the corresponding second loudness adjustment value is smaller. If the type of audio content is speech, the corresponding second loudness adjustment value is larger. Based on this, a correspondence between different types of audio content and second loudness adjustment values ​​is set. After determining the type of audio content of the media data to be published, the corresponding second loudness adjustment value can be determined using the correspondence.

[0078] After the second loudness adjustment value is determined, the target loudness can be determined by increasing or decreasing the corresponding second loudness adjustment value based on the preset published loudness.

[0079] The type of audio content can ensure the type of media data to be published, and can be used to determine the corresponding second loudness adjustment, thereby further improving the reliability of the target loudness.

[0080] Step S3023: Determine loudness equalization parameters corresponding to complete loudness equalization based on the target loudness and the current loudness in the audio feature.

[0081] As described above, the loudness equalization parameter corresponding to the complete loudness equalization includes gain. Based on this, in the loudness equalization mode of complete loudness equalization, gain=the difference between the target loudness and the current loudness.

[0082] In some optional implementations, step S3023 includes: calculating a difference between the target loudness and the current loudness, and determining the difference as a first gain corresponding to complete loudness equalization, and the loudness equalization parameter includes the first gain.

[0083] In order to distinguish it from the gain corresponding to the subsequent partial loudness equalization, the gain corresponding to the complete loudness equalization is referred to as the first gain. For example, if the target loudness is -18 and the current loudness is -22, the first gain = -18 - (-22) = 4.

[0084] On the basis of the obtained target loudness with higher reliability, the first gain is obtained by utilizing the difference between the target loudness and the current loudness, thereby ensuring the accuracy of the first gain.

[0085] Step S3024: If the loudness equalization mode is partial loudness equalization, a second gain corresponding to the partial loudness equalization is determined based on the current peak value and the preset peak value in the audio feature.

[0086] The loudness equalization parameters include a preset peak value and a second gain.

[0087] The second gain may be the difference between the preset peak value and the current peak value, for example, if the preset peak value is 0 dbfs and the current peak value is -1 dbfs, the second gain = 0 - (-1) = 1. Under partial loudness equalization, the loudness equalization parameters include the preset peak value and the second gain.

[0088] Step S303: Based on the loudness equalization method and loudness equalization parameters, the audio of the media data to be published is subjected to loudness equalization processing to determine the published media data. Figure 2 The detailed description of step S203 of the illustrated embodiment is not repeated here.

[0089] The audio processing method provided in this embodiment, when the loudness equalization mode is complete loudness equalization, determines the target loudness of the media data to be published through audio features, and then determines the loudness equalization parameters in combination with the current loudness. Since the loudness equalization parameters are obtained based on the audio features, the reliability of the obtained loudness equalization parameters is improved, thereby ensuring the sound quality of the published media data obtained after the loudness equalization processing. When the loudness equalization mode is partial loudness equalization, the second gain is determined using the current peak value and the preset peak value, and the second gain and the preset peak value are used as the loudness equalization parameters, thereby ensuring the accuracy of the obtained loudness equalization parameters.

[0090] As a specific application embodiment of the disclosed embodiment, after the user creates a short video to be released, the user releases the short video through the mobile phone application software. After the user uploads the short video to be released through the application software, the application software performs loudness equalization processing on the short video to be released after obtaining the user's authorization. For example, Figure 4 As shown, the audio features of the short video to be released are extracted, and the dynamic range corresponding to the short video to be released is compared with the dynamic range threshold of 24, or the proportion of sampling points exceeding 0dbfs in the short screen to be released is compared with the proportion threshold of 5%. If the dynamic range>24, or the proportion of sampling points exceeding 0dbfs is <5%, the loudness equalization method is determined to be full loudness equalization; otherwise, the loudness equalization method is determined to be partial loudness equalization. If the loudness equalization method is full loudness equalization, the target loudness (e.g., -18) is determined based on the type of audio content or the voice proportion, and then the first gain gain=-18-(-22)=4 is calculated based on the target loudness. If the loudness equalization method is partial loudness equalization, a preset peak value (e.g., 0dbfs) is obtained and a second gain gain=0-(-1)=1 is calculated. After determining the loudness equalization method and the loudness equalization parameters, the short video to be released is subjected to loudness equalization processing to obtain a released short video, which can then be released.

[0091] In this embodiment, an audio processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0092] This embodiment provides an audio processing device, such as Figure 5 As shown, including:

[0093] The audio feature extraction module 501 is used to extract audio features of the media data to be published.

[0094] The loudness equalization determination module 502 is used to determine the loudness equalization method and loudness equalization parameters of the media data to be published based on the magnitude relationship between the audio feature and the corresponding threshold.

[0095] The loudness equalization processing module 503 is used to perform loudness equalization processing on the audio of the media data to be published based on the loudness equalization mode and the loudness equalization parameters, and determine to publish the media data.

[0096] In some optional implementations, the loudness equalization parameter determination method in the loudness equalization determination module 502 includes:

[0097] The target loudness determination unit is configured to determine the target loudness of the to-be-published media data based on the audio feature if the loudness equalization mode is complete loudness equalization.

[0098] The loudness equalization determination unit is used to determine the loudness equalization parameters corresponding to the complete loudness equalization based on the target loudness and the current loudness in the audio feature.

[0099] In some optional implementations, the target loudness determination unit includes:

[0100] The loudness acquisition subunit is used to acquire the preset release loudness.

[0101] The target loudness determination subunit is used to determine the target loudness based on the audio feature and the preset release loudness.

[0102] In some optional implementations, the audio feature includes a speech ratio, and the target loudness determination subunit includes:

[0103] The first loudness adjustment value determining subunit is configured to determine a first loudness adjustment value based on the speech proportion.

[0104] The first adjustment subunit is configured to adjust the preset published loudness based on the first loudness adjustment value to determine the target loudness.

[0105] In some optional implementations, the audio feature includes the type of audio content, and the target loudness determination subunit includes:

[0106] The second loudness adjustment value obtaining subunit is configured to obtain a second loudness adjustment value corresponding to the type of audio content.

[0107] The second adjustment subunit is configured to adjust the preset published loudness based on the second loudness adjustment value to determine the target loudness.

[0108] In some optional implementations, the loudness equalization determination unit includes:

[0109] The calculation subunit is used to calculate the difference between the target loudness and the current loudness, and determine the difference as a first gain corresponding to the complete loudness equalization, and the loudness equalization parameter includes the first gain.

[0110] In some optional implementations, the loudness equalization parameter determination method in the loudness equalization determination module 502 includes:

[0111] The partial loudness equalization unit is used to determine a second gain corresponding to the partial loudness equalization based on a current peak value and a preset peak value in the audio feature if the loudness equalization mode is partial loudness equalization, and the loudness equalization parameters include the preset peak value and the second gain.

[0112] In some optional implementations, the loudness equalization determination module 502 includes:

[0113] The first loudness equalization mode determining unit is configured to determine the loudness equalization mode based on a magnitude relationship between the dynamic range and a dynamic range threshold if the audio feature includes a dynamic range.

[0114] The second loudness equalization mode determining unit is used to count the proportion of loudness sampling points whose loudness exceeds a preset peak value if the audio feature includes loudness sampling points, and determine the loudness equalization mode based on the relationship between the proportion and the proportion threshold.

[0115] The audio processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0116] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0117] The present disclosure also provides a computer device having the above Figure 5 The audio processing device shown.

[0118] See also Figure 6 , Figure 6 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0119] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0120] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0121] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0123] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0124] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0125] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0126] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An audio processing method, characterized in that: The method comprises: Extracting audio features of media data to be published; Determining a loudness equalization method and loudness equalization parameters of the to-be-published media data based on a magnitude relationship between the audio feature and the corresponding threshold; Based on the loudness equalization mode and the loudness equalization parameters, loudness equalization processing is performed on the audio of the media data to be published, and the media data to be published is determined.

2. The method according to claim 1, characterized in that The method for determining the loudness equalization parameter includes: If the loudness equalization mode is complete loudness equalization, determining a target loudness of the to-be-published media data based on the audio feature; Based on the target loudness and the current loudness in the audio feature, a loudness equalization parameter corresponding to the complete loudness equalization is determined.

3. The method according to claim 2, characterized in that The determining the target loudness of the to-be-published media data based on the audio feature includes: Get the preset release loudness; The target loudness is determined based on the audio feature and the preset release loudness.

4. The method according to claim 3, characterized in that The audio feature includes a voice ratio, and determining the target loudness based on the audio feature and the preset release loudness includes: Determining a first loudness adjustment value based on the voice proportion; The preset published loudness is adjusted based on the first loudness adjustment value to determine the target loudness.

5. The method according to claim 3, characterized in that: The audio feature includes a type of audio content, and determining the target loudness based on the audio feature and the preset release loudness includes: Obtaining a second loudness adjustment value corresponding to the type of the audio content; The preset published loudness is adjusted based on the second loudness adjustment value to determine the target loudness.

6. The method according to claim 2, characterized in that The determining, based on the target loudness and the current loudness in the audio feature, a loudness equalization parameter corresponding to the complete loudness equalization includes: A difference between the target loudness and the current loudness is calculated, and the difference is determined as a first gain corresponding to the complete loudness equalization, wherein the loudness equalization parameter includes the first gain.

7. The method according to claim 1, characterized in that The method for determining the loudness equalization parameter includes: If the loudness equalization mode is partial loudness equalization, a second gain corresponding to the partial loudness equalization is determined based on a current peak value and a preset peak value in the audio feature, and the loudness equalization parameters include the preset peak value and the second gain.

8. The method according to any one of claims 1 to 7, characterized in that The determining, based on the magnitude relationship between the audio feature and the corresponding threshold, a loudness balancing method of the to-be-published media data includes: If the audio feature includes a dynamic range, determining the loudness equalization method based on a magnitude relationship between the dynamic range and a dynamic range threshold; If the audio feature includes loudness sampling points, the proportion of loudness sampling points whose loudness exceeds a preset peak value is counted, and the loudness equalization method is determined based on a magnitude relationship between the proportion and a proportion threshold.

9. An audio processing device, characterized in that: The device comprises: An audio feature extraction module, used to extract audio features of the media data to be published; A loudness equalization determination module, configured to determine a loudness equalization method and loudness equalization parameters of the to-be-published media data based on a magnitude relationship between the audio feature and a corresponding threshold; The loudness equalization processing module is used to perform loudness equalization processing on the audio of the media data to be published based on the loudness equalization method and the loudness equalization parameters, and determine to publish the media data.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the audio processing method according to any one of claims 1 to 8 by executing the computer instructions.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the audio processing method according to any one of claims 1 to 8.

Images