Audio compression model processing method and system, equipment and program product

Abstract

The invention discloses an audio compression model processing method and system, equipment and a program product, and the method comprises the steps: compressing a sample audio through an initial audio compression model, and obtaining a predicted audio; respectively dividing the sample audio and the predicted audio into a sensitive frequency band and a non-sensitive frequency band; dividing sensitive frequency bands of the sample audio and the predicted audio into four frequency bands based on a first auditory masking threshold value, and determining a weight corresponding to a frequency spectrum square difference value of the frequency bands in the same range of the sample audio and the predicted audio, dividing the non-sensitive evaluation of the sample audio and the predicted audio into four frequency bands based on a second auditory masking threshold, and determining a weight corresponding to a frequency spectrum square difference value of the frequency bands of the same range of the sample audio and the predicted audio; obtaining the frequency spectrum Mel sub-band loss by using each frequency spectrum square difference value and the corresponding weight; and optimizing the initial audio compression model based on the spectrum Mel sub-band loss to obtain a target audio compression model.

Description

Technical Field

[0001] This application relates to the field of audio data processing technology, and in particular to a method, system, device, and program product for processing an audio compression model. Background Technology

[0002] Since the original audio data is usually large in size, audio data encoding and decoding has always been an important research branch in the audio field in order to reduce the amount of data and facilitate the transmission and storage of audio data.

[0003] Traditional audio encoding and decoding relies on signal processing and psychological models for compression, employing lossy coding compression driven by fixed rules. Specifically, these rules compress useless audio data by discarding certain bits, thus compressing the audio data to some extent. However, applying these fixed rules to different types of audio produces inconsistent results, easily retaining a large amount of useless data or compressing useful data. Artificial intelligence audio compression models (AI codecs), on the other hand, are data-driven encoding and decoding models. They process audio data using an audio compression model, analyze the loss before and after processing, and update the audio compression model based on the resulting loss. This allows them to achieve better compression ratios and fidelity compared to traditional methods.

[0004] However, in existing audio compression models, the frequent loss of data before and after encoder processing is calculated by directly using the input audio and the data processed by the encoder. That is, all information is considered and processed in a unified manner to avoid the loss of effective information, but this will affect the performance of the model and the compression ratio that can be achieved is relatively limited. Summary of the Invention

[0005] In view of the shortcomings of the prior art, this application provides a method, system, device and program product for processing audio compression model, so as to solve the problem that the prior art cannot guarantee that effective messages are not lost, and thus fully compresses audio data.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] The first aspect of this application provides a method for processing an audio compression model, including:

[0008] The sample audio is compressed using an initial audio compression model to obtain the predicted audio.

[0009] The sample audio and the predicted audio are divided into frequency bands to obtain sample sensitive frequency bands, sample non-sensitive frequency bands, predicted sensitive frequency bands, and predicted non-sensitive frequency bands.

[0010] The sample sensitive frequency band and the predicted sensitive frequency band are divided based on the first auditory masking threshold to obtain the first sample sensitive frequency band and the second sample sensitive frequency band, as well as the first predicted sensitive frequency band and the second predicted sensitive frequency band.

[0011] Based on the second auditory masking threshold, the sample non-sensitive frequency band and the predicted non-sensitive frequency band are divided to obtain the first sample non-sensitive frequency band and the second sample non-sensitive frequency band, as well as the first predicted non-sensitive frequency band and the second predicted non-sensitive frequency band.

[0012] Calculate the squared difference of the first sub-band spectrum between the first sample sensitive frequency band and the first predicted sensitive frequency band, the squared difference of the second sub-band spectrum between the second sample sensitive frequency band and the second predicted sensitive frequency band, the squared difference of the third sub-band spectrum between the first sample non-sensitive frequency band and the first predicted non-sensitive frequency band, and the squared difference of the fourth sub-band spectrum between the second sample non-sensitive frequency band and the second predicted non-sensitive frequency band, respectively.

[0013] Determine the weights corresponding to the squared differences of the first sub-band spectrum, the second sub-band spectrum, the third sub-band spectrum, and the fourth sub-band spectrum.

[0014] Based on the squared difference of the first sub-band spectrum and its corresponding weight, the squared difference of the second sub-band spectrum and its corresponding weight, the squared difference of the third sub-band spectrum and its corresponding weight, and the squared difference of the fourth sub-band spectrum and its corresponding weight, the Mel sub-band spectrum loss is obtained.

[0015] The initial audio compression model is optimized based on the spectral Mel subband loss to obtain the target audio compression model.

[0016] Optionally, in the above-described audio compression model processing method, the step of dividing the sample audio and the predicted audio into frequency bands to obtain sample sensitive frequency bands, sample non-sensitive frequency bands, predicted sensitive frequency bands, and predicted non-sensitive frequency bands includes:

[0017] Perform short-time Fourier transforms on the sample audio and the predicted audio respectively;

[0018] The Mel subband frequency bands of the transformed sample audio are divided into sample sensitive frequency bands and sample non-sensitive frequency bands according to the human ear sensitive frequency bands, and the Mel subband frequency bands in the transformed predicted audio are divided into prediction sensitive frequency bands and prediction non-sensitive frequency bands according to the human ear sensitive frequency bands; wherein, the human ear sensitive frequency bands are from 2 kHz to 5 kHz.

[0019] Optionally, in the above-described audio compression model processing method, determining the weights corresponding to the squared differences of the first sub-band spectrum, the second sub-band spectrum, the third sub-band spectrum, and the fourth sub-band spectrum includes:

[0020] Based on the power value of the first sample sensitive frequency band, a first weighting function is used to calculate the weight, thereby obtaining the weight corresponding to the squared difference of the first sub-band spectrum; and based on the power value of the second sample sensitive frequency band, a second weighting function is used to calculate the weight, thereby obtaining the weight corresponding to the squared difference of the second sub-band spectrum; wherein, the slope of the first weighting function is less than the slope of the second weighting function;

[0021] Based on the power values ​​of the first sample's non-sensitive frequency bands, a third weighting function is used to calculate the weights, resulting in the weights corresponding to the squared differences in the spectrum of the third sub-band. Similarly, based on the power values ​​of the second sample's non-sensitive frequency bands, a fourth weighting function is used to calculate the weights, resulting in the weights corresponding to the squared differences in the spectrum of the fourth sub-band. The slope of the third weighting function is less than the slope of the first weighting function; the slope of the fourth weighting function is less than the slope of the second weighting function and greater than the slope of the third weighting function.

[0022] Optionally, in the above-described audio compression model processing method, the step of calculating weights based on the power values ​​of the first sample sensitive frequency band using a first weighting function to obtain the weights corresponding to the squared difference of the first sub-band spectrum, and calculating weights based on the power values ​​of the second sample sensitive frequency band using a second weighting function to obtain the weights corresponding to the squared difference of the second sub-band spectrum, includes:

[0023] The weights corresponding to the squared differences of the first sub-band spectrum are obtained by calculating twice the natural logarithm to the power of M using the first weighting function; where M is the power value of the non-sensitive frequency band of the first sample.

[0024] The weights corresponding to the squared difference of the second sub-band spectrum are obtained by calculating twice the natural logarithm to the power of N using the second weighting function; where N is twice the power value of the sensitive frequency band of the second sample.

[0025] Optionally, in the above-described audio compression model processing method, the step of calculating the weights based on the power values ​​of the first sample's non-sensitive frequency bands using a third weighting function to obtain the weights corresponding to the squared difference of the third sub-band spectrum, and calculating the weights based on the power values ​​of the second sample's non-sensitive frequency bands using a fourth weighting function to obtain the weights corresponding to the squared difference of the fourth sub-band spectrum, includes:

[0026] The weights corresponding to the squared difference of the third sub-band spectrum are obtained by calculating the natural logarithm to the power of K using the third weighting function; where K is the power value of the non-sensitive frequency band of the first sample.

[0027] The weights corresponding to the squared difference of the fourth sub-band spectrum are obtained by calculating the L-th power of the natural object through the fourth weighting function; where L is twice the power value of the second sample non-sensitive frequency band.

[0028] Optionally, the above-described audio compression model processing method further includes:

[0029] The power values ​​of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band are calculated based on the complex amplitudes of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band, respectively.

[0030] Optionally, in the above-described audio compression model processing method, the step of calculating the power value of each audio signal based on the complex amplitude of each audio signal in the sample sensitive frequency band, the sample non-sensitive frequency band, the predicted sensitive frequency band, and the predicted non-sensitive frequency band includes:

[0031] For each target sample frequency band, the amplitude ratio of the target sample frequency band to the base-10 factor of 20 is calculated to obtain the power value of the target sample frequency band; wherein, the target sample frequency band includes the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band; the amplitude ratio of the target sample frequency band is the ratio of the absolute value of the complex amplitude of the target sample frequency band to a reference reference; the reference reference is the maximum value of the complex amplitude of all Mel subband frequency bands in the sample audio.

[0032] A second aspect of this application provides a processing system for an audio compression model, comprising:

[0033] The compression processing unit is used to compress the sample audio using the initial audio compression model to obtain the predicted audio;

[0034] An audio segmentation unit is used to divide the sample audio and the predicted audio into frequency bands respectively, to obtain sample sensitive frequency bands, sample non-sensitive frequency bands, predicted sensitive frequency bands, and predicted non-sensitive frequency bands;

[0035] A sensitive frequency band division unit is used to divide the sample sensitive frequency band and the predicted sensitive frequency band based on a first auditory masking threshold, to obtain a first sample sensitive frequency band and a second sample sensitive frequency band, as well as a first predicted sensitive frequency band and a second predicted sensitive frequency band.

[0036] The non-sensitive frequency band division unit is used to divide the sample non-sensitive frequency band and the predicted non-sensitive frequency band based on a second auditory masking threshold, to obtain a first sample non-sensitive frequency band and a second sample non-sensitive frequency band, as well as a first predicted non-sensitive frequency band and a second predicted non-sensitive frequency band.

[0037] The deviation calculation unit is used to calculate the squared difference of the first sub-band spectrum between the first sample sensitive frequency band and the first predicted sensitive frequency band, the squared difference of the second sub-band spectrum between the second sample sensitive frequency band and the second predicted sensitive frequency band, the squared difference of the third sub-band spectrum between the first sample non-sensitive frequency band and the first predicted non-sensitive frequency band, and the squared difference of the fourth sub-band spectrum between the second sample non-sensitive frequency band and the second predicted non-sensitive frequency band.

[0038] The weight analysis unit is used to determine the weights corresponding to the squared differences of the first sub-band spectrum, the second sub-band spectrum, the third sub-band spectrum, and the fourth sub-band spectrum.

[0039] The sample audio weighting unit is used to obtain the spectral Mel subband loss based on the first subband spectral squared difference and its corresponding weight, the second subband spectral squared difference and its corresponding weight, the third subband spectral squared difference and its corresponding weight, and the fourth subband spectral squared difference and its corresponding weight.

[0040] The model optimization unit is used to optimize the initial audio compression model based on the spectral Mel subband loss to obtain the target audio compression model.

[0041] Optionally, in the above-described audio compression model processing system, the audio partitioning unit includes:

[0042] The transformation unit is used to perform short-time Fourier transforms on the sample audio and the predicted audio, respectively.

[0043] The frequency band division unit is used to divide the Mel subband frequency band of the transformed sample audio into sample sensitive frequency band and sample non-sensitive frequency band according to the human ear sensitive frequency band, and to divide the Mel subband frequency band of the transformed predicted audio into prediction sensitive frequency band and prediction non-sensitive frequency band according to the human ear sensitive frequency band; wherein, the human ear sensitive frequency band is 2 kHz to 5 kHz.

[0044] Optionally, in the above-described audio compression model processing system, the weight analysis unit includes:

[0045] The first weight calculation unit is used to calculate the weight based on the power value of the first sample sensitive frequency band using a first weight function to obtain the weight corresponding to the squared difference of the first sub-band spectrum, and to calculate the weight based on the power value of the second sample sensitive frequency band using a second weight function to obtain the weight corresponding to the squared difference of the second sub-band spectrum; wherein, the slope of the first weight function is less than the slope of the second weight function;

[0046] The second weight calculation unit is used to calculate the weight based on the power value of the non-sensitive frequency band of the first sample using a third weight function to obtain the weight corresponding to the squared difference of the spectrum of the third sub-band, and to calculate the weight based on the power value of the non-sensitive frequency band of the second sample using a fourth weight function to obtain the weight corresponding to the squared difference of the spectrum of the fourth sub-band; wherein, the slope of the third weight function is less than the slope of the first weight function; the slope of the fourth weight function is less than the slope of the second weight function and greater than the slope of the third weight function.

[0047] Optionally, in the above-described audio compression model processing system, the first weight calculation unit includes:

[0048] The first calculation unit is used to calculate twice the natural logarithm to the power of M through the first weighting function to obtain the weight corresponding to the squared difference of the first sub-band spectrum; where M is the power value of the non-sensitive frequency band of the first sample.

[0049] The second calculation unit is used to calculate the weight corresponding to the squared difference of the second sub-band spectrum by power N, which is twice the natural logarithm, through the second weighting function; where N is twice the power value of the second sample sensitive frequency band.

[0050] Optionally, in the above-described audio compression model processing method, the second weight calculation unit includes:

[0051] The third calculation unit is used to calculate the natural logarithm to the power of K through the third weighting function to obtain the weight corresponding to the squared difference of the spectrum of the third sub-band; where K is the power value of the non-sensitive frequency band of the first sample;

[0052] The fourth calculation unit is used to calculate the L-th power of the natural object through the fourth weighting function to obtain the weight corresponding to the squared difference of the fourth sub-band spectrum; where L is twice the power value of the second sample non-sensitive frequency band.

[0053] Optionally, the processing system of the above-mentioned audio compression model also includes:

[0054] The power value calculation unit is used to calculate the power values ​​of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band based on the complex amplitudes of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band, respectively.

[0055] Optionally, in the above-described audio compression model processing method, the power value calculation unit includes:

[0056] The power value calculation subunit is used to calculate the amplitude ratio of the target sample frequency band to base 10 by a factor of 20 for each target sample frequency band, thereby obtaining the power value of the target sample frequency band; wherein, the target sample frequency band includes the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band; the amplitude ratio of the target sample frequency band is the ratio of the absolute value of the complex amplitude of the target sample frequency band to a reference reference; the parameter reference is the maximum value of the complex amplitude of all Mel subband frequency bands in the sample audio.

[0057] A third aspect of this application provides a computer device, including a memory and a processor;

[0058] The memory is used to store computer programs;

[0059] The processor is used to execute the computer program to implement the processing method of the audio compression model as described in any of the above.

[0060] The fourth aspect of this application provides a computer program product including executable computer instructions;

[0061] When the computer instructions are executed by the processor, they are used to implement the processing method of the audio compression model as described in any of the above.

[0062] This application provides a method for processing an audio compression model. An initial audio compression model compresses sample audio to obtain predicted audio. The sample audio and predicted audio are then divided into frequency bands: a sample sensitive frequency band, a sample non-sensitive frequency band, a predicted sensitive frequency band, and a predicted non-sensitive frequency band. This divides the two audio samples into two parts: one sensitive to human hearing and the other non-sensitive to human hearing. This allows for separate processing of these two parts, making the model more focused on audio signals within the sensitive frequency band—that is, focusing on more effective audio data that the human ear can easily and clearly perceive—avoiding over-compression and loss of effective information, while still ensuring sufficient compression of audio signals within the sensitive frequency band. Then, based on a first auditory masking threshold, a first and second auditory sensitive weights are determined for the sample sensitive frequency band and the predicted sensitive frequency band. Similarly, based on a second auditory masking threshold, a first and second non-auditory sensitive weights are determined for the sample non-sensitive frequency band and the predicted non-auditory non-sensitive frequency band. This allows for the allocation of greater weights to more effective audio signals and smaller weights to relatively less important audio signals through the set auditory masking thresholds. Then, based on the first auditory masking threshold, the sample sensitive frequency band and the predicted sensitive frequency band are divided into the first sample sensitive frequency band and the second sample sensitive frequency band, as well as the first predicted sensitive frequency band and the second predicted sensitive frequency band. Similarly, based on the second auditory masking threshold, the sample insensitive frequency band and the predicted insensitive frequency band are divided into the first sample insensitive frequency band and the second sample insensitive frequency band, as well as the first predicted insensitive frequency band and the second predicted insensitive frequency band. This further divides the auditory masked and unmasked frequency bands to allow for targeted analysis of their losses. Then, the squared difference of the first sub-band spectrum between the first sample sensitive frequency band and the first predicted sensitive frequency band, the squared difference of the second sub-band spectrum between the second sample sensitive frequency band and the second predicted sensitive frequency band, the squared difference of the third sub-band spectrum between the first sample insensitive frequency band and the first predicted insensitive frequency band, and the squared difference of the fourth sub-band spectrum corresponding to the second sample insensitive frequency band and the second predicted insensitive frequency band are calculated respectively. This determines the initial spectral error of each frequency band, which is used for subsequent calculation of the loss of each frequency band. Next, the weights corresponding to the squared differences of the first, second, third, and fourth sub-band spectra are determined. Based on the squared differences of the first, second, third, and fourth sub-band spectra and their corresponding weights, the Mel sub-band loss is obtained. By calculating the corresponding weights for each frequency band, the spectrum is weighted, thus giving more attention to the data in important frequency bands while avoiding overemphasizing the data in relatively unimportant frequency bands. This effectively avoids the loss of useful information and allows for sufficient compression of unimportant data.Finally, the initial audio compression model is optimized based on the spectral Mel subband loss to obtain the target audio compression model. This target audio compression model can avoid the loss of effective information and fully compress the audio to achieve the best compression ratio. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0064] Figure 1 A flowchart illustrating a processing method for an audio compression model provided in this application embodiment;

[0065] Figure 2 A flowchart illustrating a method for dividing sample audio and predicted audio into two frequency bands, provided in this application embodiment;

[0066] Figure 3 A logical schematic diagram of an audio compression model processing method provided in an embodiment of this application;

[0067] Figure 4 A schematic diagram of the architecture of an audio compression model processing system provided in an embodiment of this application;

[0068] Figure 5 This is a schematic diagram of the architecture of a computer device provided in an embodiment of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0071] This application provides a method for processing an audio compression model, such as... Figure 1 As shown, it includes the following steps:

[0072] S101. The sample audio is compressed using the initial audio compression model to obtain the predicted audio.

[0073] The initial audio compression model is the audio compression model that needs to be optimized, such as the audio compression model that needs to be optimized in the current training round.

[0074] Optionally, the audio compression model can be a DAC model or other models. The audio compression model mainly consists of an encoder, a quantizer, a decoder, and a discriminator. In the specific training process, the audio to be compressed is input into the encoder, which performs convolutional encoding. By controlling the stride of the convolutional encoder, downsampling compression is achieved. Then, the quantizer uses RVQ (Residual Vector Quantization) or one of FSQ, VQ, or simVQ to quantize the downsampled audio signal from the encoder into a smaller integer sequence. Next, the quantized integer sequence is input into the decoder, where deconvolution is used to gradually recover the audio signal. Then, the output audio signal after upsampling and the original audio signal are input into the decoder to make the output signal closer to the original true signal. Finally, the network loss is calculated, and the audio compression model is updated based on the network loss until the loss converges or the required number of iterations is reached.

[0075] The network losses include: Ls, the spectral loss, which measures the spectral similarity of the audio signals before and after encoding; Lt, the temporal loss, which measures the temporal similarity of the audio signals before and after encoding; Lw, the quantization loss of the quantizer, which measures the similarity between the codebook and the signal before quantization; Ll, the entropy coding loss of the quantizer, which measures the similarity of the encoded vector before and after secondary compression; and ld and lg, the discriminator and generator losses, respectively, which measure the similarity between the generated samples and the original samples.

[0076] Since the current spectral loss is calculated directly using the audio signals before and after encoder processing, it considers all information to avoid losing useful information. However, this affects model performance, and because it retains a lot of information, the achievable compression ratio is relatively limited. Therefore, in this embodiment, when spectral loss is needed, the encoder first processes the two audio data before and after processing the audio to be compressed. That is, the encoder of the audio compression model in the current training round processes the two audio data before and after processing the audio to be compressed. In other words, the predicted audio is specifically the audio signal processed by the encoder of the initial audio compression model.

[0077] S102. Divide the sample audio and the predicted audio into frequency bands respectively to obtain the sample sensitive frequency band, the sample non-sensitive frequency band, the predicted sensitive frequency band, and the predicted non-sensitive frequency band.

[0078] Among them, the sample sensitive frequency band and the sample non-sensitive frequency band are the spectra of the sample audio that are within the sensitive frequency bands of the human ear and the spectra that are not within the sensitive frequency bands of the human ear, respectively. The prediction sensitive frequency band and the prediction non-sensitive frequency band are the spectra of the predicted audio that are within the sensitive frequency bands of the human ear and the spectra that are not within the sensitive frequency bands of the human ear, respectively.

[0079] Because the human ear has sensitive frequency bands, sounds within these bands are easily and clearly heard, while sounds outside these bands are relatively difficult to hear. Therefore, when listening to audio data, users primarily listen to the audio within their sensitive frequency bands. This audio within these bands is considered more valuable, meaning it's more important. Thus, it shouldn't be compressed excessively to avoid losing valuable information. Audio outside these bands, on the other hand, has lower value and can be compressed as much as possible to fully compress the entire audio data.

[0080] Therefore, in this embodiment, the sample audio and predicted audio are converted into data, and then divided into two frequency bands according to the ear-sensitive frequency band. Specifically, the audio signals in the sample audio that are within the ear-sensitive frequency band are classified as the sample sensitive frequency band, and the remaining audio signals are classified as the sample non-sensitive frequency band. Similarly, the audio signals in the predicted audio that are within the ear-sensitive frequency band are classified as the prediction sensitive frequency band, and the remaining audio signals are classified as the prediction non-sensitive frequency band. In other words, the two audio signals are divided into audio data in sensitive frequency bands and audio data in non-sensitive frequency bands, so that these two frequency bands can be processed separately, thereby ensuring that the effective information of the audio is preserved while fully compressing the audio data.

[0081] Optionally, in another embodiment of this application, one specific implementation of step S102 is as follows: Figure 2 As shown, it includes:

[0082] S201. Perform short-time Fourier transform on the sample audio and the predicted audio respectively.

[0083] In order to determine the frequency domain of the audio so that it can be divided according to frequency bands, a short-time Fourier transform is performed on the sample audio and the predicted audio to convert them into the Mel spectrum, which is composed of individual Mel subband frequency bands, i.e., converted into individual smallest units mel_bin.

[0084] S202. Divide the Mel subband frequency bands in the transformed sample audio into sample sensitive frequency bands and sample non-sensitive frequency bands according to the frequency bands sensitive to the human ear, and divide the Mel subband frequency bands in the transformed predicted audio into prediction sensitive frequency bands and prediction non-sensitive frequency bands according to the frequency bands sensitive to the human ear.

[0085] The human ear is sensitive to frequencies ranging from 2 kHz to 5 kHz. Therefore, the sample audio and predicted audio after short-time Fourier transform are divided into two parts according to frequency band: 2 kHz-5 kHz and <2 kHz or >5 kHz. That is, the 2 kHz-5 kHz portion of the sample audio is classified as the sample sensitive frequency band, and the <2 kHz and >5 kHz portions are classified as the sample non-sensitive frequency band. Similarly, the 2 kHz-5 kHz portion of the predicted audio is classified as the prediction sensitive frequency band, and the <2 kHz and >5 kHz portions are classified as the prediction non-sensitive frequency band.

[0086] S103. Based on the first auditory masking threshold, the sample sensitive frequency band and the predicted sensitive frequency band are divided to obtain the first sample sensitive frequency band and the second sample sensitive frequency band, as well as the first predicted sensitive frequency band and the second predicted sensitive frequency band.

[0087] It should be noted that, since the difficulty of human ears capturing different parts of the Mel subband frequency band within the sensitive frequency band varies, in order to make the audio compression model pay more attention to the more important Mel subband frequency band within the sensitive frequency band, so as to avoid the loss of effective information, while also allowing for adaptive compression of the data in the relatively unimportant Mel subband frequency band.

[0088] Therefore, an appropriate auditory masking threshold is set for the frequency bands sensitive to human hearing, namely the first auditory masking threshold. The frequency bands sensitive to human hearing are divided into different parts, and different weights are assigned to the frequency bands above and below the first auditory masking threshold.

[0089] Specifically, the portion of the sample sensitive frequency band below the first auditory masking threshold is designated as the first sample sensitive frequency band, and the portion of the sample sensitive frequency band above the first auditory masking threshold is designated as the second sample sensitive frequency band. Similarly, the portion of the prediction sensitive frequency band below the first auditory masking threshold is designated as the first prediction sensitive frequency band, and the portion of the prediction sensitive frequency band above the first auditory masking threshold is designated as the second prediction sensitive frequency band.

[0090] S104. Based on the second auditory masking threshold, the sample non-sensitive frequency band and the predicted non-sensitive frequency band are divided to obtain the first sample non-sensitive frequency band and the second sample non-sensitive frequency band, as well as the first predicted non-sensitive frequency band and the second predicted non-sensitive frequency band.

[0091] Similarly, the ease with which different parts of the Mel subband in non-human ear sensitive frequency bands are captured by the human ear also varies. Therefore, in order to make the audio compression model pay more attention to the more important Mel subband in non-human ear sensitive frequency bands, so as to avoid the loss of effective information, while fully compressing the data of the relatively unimportant Mel subband.

[0092] Therefore, a second auditory masking threshold is set to divide the sample insensitive frequency band and the prediction insensitive frequency band separately. Specifically, the portion of the sample insensitive frequency band below the second auditory masking threshold is classified as the first sample insensitive frequency band, and the portion of the sample insensitive frequency band above the second auditory masking threshold is classified as the second sample insensitive frequency band. Similarly, the portion of the prediction insensitive frequency band below the second auditory masking threshold is classified as the first prediction insensitive frequency band, and the portion of the prediction insensitive frequency band above the second auditory masking threshold is classified as the second prediction insensitive frequency band.

[0093] Optionally, for audio data in the frequency band sensitive to human hearing, i.e., audio data in the 2kHz-5kHz frequency band, the first auditory masking threshold is set to -80dB. For audio data in non-sensitive frequency bands, the second auditory masking threshold is set to -40dB.

[0094] It should be noted that the frequency band division based on the first auditory masking threshold and the second auditory masking threshold are independent of each other, so steps S103 and S104 can be executed simultaneously, or one of the steps can be executed first.

[0095] S105. Calculate the squared difference of the first sub-band spectrum between the first sample sensitive frequency band and the first predicted sensitive frequency band, the squared difference of the second sub-band spectrum between the second sample sensitive frequency band and the second predicted sensitive frequency band, the squared difference of the third sub-band spectrum between the first sample non-sensitive frequency band and the first predicted non-sensitive frequency band, and the squared difference of the fourth sub-band spectrum between the second sample non-sensitive frequency band and the second predicted non-sensitive frequency band.

[0096] It should be noted that in order to optimize the initial audio compression model, it is necessary to analyze the spectral deviation between the sample audio before compression and the predicted audio after compression. This allows us to determine the model loss based on the spectral deviation, and then optimize the initial audio compression model based on the model loss.

[0097] Since the sample audio and predicted audio are divided into multiple frequency bands in this embodiment, it is necessary to calculate the spectral deviation of the same frequency band of the sample audio and predicted audio respectively, so as to determine the spectral loss of each frequency band.

[0098] Specifically, the square of the difference between the spectrum within the first sample sensitive frequency band and the spectrum within the first predicted sensitive frequency band, both within the same frequency band range, is calculated to obtain the squared difference of the first sub-band spectrum. The square of the difference between the spectrum within the second sample sensitive frequency band and the spectrum within the second predicted sensitive frequency band, both within the same frequency band range, is calculated to obtain the squared difference of the second sub-band spectrum. The square of the difference between the spectrum within the first sample non-sensitive frequency band and the spectrum within the first predicted non-sensitive frequency band, both within the same frequency band range, is calculated to obtain the squared difference of the third sub-band spectrum. The square of the difference between the spectrum within the second sample non-sensitive frequency band and the spectrum within the second predicted non-sensitive frequency band, both within the same frequency band range, is calculated to obtain the squared difference of the fourth sub-band spectrum.

[0099] S106. Determine the weights corresponding to the squared differences of the first sub-band spectrum, the second sub-band spectrum, the third sub-band spectrum, and the fourth sub-band spectrum.

[0100] It should be noted that the squared differences of the first, second, third, and fourth sub-band spectra are the squared differences of the spectra of each frequency band. Since the importance of each frequency band varies, to account for the loss based on the importance of each band, a weight is determined for each squared difference of the spectrum of each sub-band, and the squared differences of the spectrum of each band are weighted accordingly.

[0101] Optionally, since the higher the power value, the easier it is for the human ear to detect, the weights corresponding to the squared differences of the sub-band spectra of each frequency band can be calculated based on the intensity values ​​of each frequency band.

[0102] Since the power values ​​of the second sample sensitive frequency band and the second predicted sensitive frequency band are above the first auditory masking threshold and are not considered auditory masking frequency bands, they are more important. Therefore, a relatively large weight is assigned to the squared difference of the second sub-band spectrum of this frequency band, resulting in the weight corresponding to the squared difference of the second sub-band spectrum. Conversely, the power values ​​of the first sample sensitive frequency band and the first predicted sensitive frequency band are below the first auditory masking threshold and are considered auditory masking frequency bands, thus their importance is lower. Therefore, a relatively small weight is assigned to the squared difference of the first sub-band spectrum of this frequency band, resulting in the weight corresponding to the squared difference of the first sub-band spectrum. Therefore, when the power values ​​are the same, the weight corresponding to the squared difference of the spectrum of mel_bin above the first auditory masking threshold is greater than the weight corresponding to the squared difference of the spectrum of mel_bin below the first auditory masking threshold.

[0103] Optionally, the relative importance of each mel_bin above and below the first auditory masking threshold to the human ear differs. Therefore, to achieve more precise compression for different mel_bins, the weights assigned to the spectral squared differences of different mel_bins vary. Specifically, the weights corresponding to the spectral squared differences of each first sub-band and each second sub-band are different. The higher the power value of a mel_bin, the greater the weight of its spectral squared difference.

[0104] While non-sensitive frequency bands are outside the range of human hearing, meaning they are relatively difficult for the human ear to detect, they are not entirely inaudible. Therefore, they also contain valid audio signals. Directly removing or compressing all non-sensitive frequency bands (i.e., predicted sensitive and predicted non-sensitive bands) would result in the loss of valuable information. Thus, they also require processing.

[0105] Similarly, since the power values ​​of the second sample insensitive frequency band and the second predicted insensitive frequency band are above the second auditory masking threshold, they are not considered auditory masking audio signals and are therefore more important. Thus, a relatively large weight is assigned to the fourth sub-band spectral squared difference value for this part, resulting in the weight corresponding to the fourth sub-band spectral squared difference value. Conversely, the power values ​​of the first sample insensitive frequency band and the first predicted insensitive frequency band are below the second auditory masking threshold, and are considered auditory masking audio signals. Therefore, their importance is lower, and a relatively small weight is assigned to the third sub-band spectral squared difference value for this part, resulting in the weight corresponding to the third sub-band spectral squared difference value. Therefore, the weight corresponding to the spectral squared difference value of mel_bin above the second auditory masking threshold is greater than the weight corresponding to the spectral squared difference value of mel_bin below the second auditory masking threshold. Optionally, the weight corresponding to different third sub-band spectral squared difference values ​​can be different, and the weight corresponding to different fourth sub-band spectral squared difference values ​​can also be different. Specifically, the larger the power value of mel_bin, the larger the weight corresponding to its spectral squared difference value.

[0106] Since the frequency bands that are sensitive to human ears are more important than those that are not, when the power values ​​of the frequency bands are the same, the weight corresponding to the square difference of the first sub-band spectrum and the square difference of the second sub-band spectrum are much greater than the weight corresponding to the square difference of the third sub-band spectrum and the square difference of the fourth sub-band spectrum.

[0107] Since it is necessary to calculate the weight of the squared difference of the frequency band based on the power value of the frequency band, optionally, in another embodiment of this application, before performing step S106, the following is further performed:

[0108] Based on the complex amplitudes of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band, the power values ​​of the two frequency bands are calculated respectively. That is, based on the complex amplitude of each mel_bin of the sample audio, the power value of each mel_bin is calculated, and this power value is used to calculate the weight corresponding to the squared difference of the spectral values ​​of each mel_bin of the sample audio and the mel_bin of the predicted audio in the same frequency band.

[0109] Specifically, the power value of the first sample sensitive frequency band is calculated based on the first sample sensitive frequency band. The power value of the second sample sensitive frequency band is calculated based on the second sample sensitive frequency band. The power value of the first sample non-sensitive frequency band is calculated based on the first sample non-sensitive frequency band. The power value of the second sample non-sensitive frequency band is calculated based on the second sample non-sensitive frequency band. It should be noted that the power value within the same frequency band range is the same. Therefore, it is also possible to calculate the power values ​​of the first predicted sensitive frequency band, the second predicted sensitive frequency band, the first predicted non-sensitive frequency band, and the second predicted non-sensitive frequency band based on the complex amplitudes of the first predicted sensitive frequency band, the second predicted sensitive frequency band, the first predicted non-sensitive frequency band, and the second predicted non-sensitive frequency band, respectively. These values ​​are then used to calculate the weights corresponding to the squared differences of the first sub-band spectrum, the second sub-band spectrum, the third sub-band spectrum, and the fourth sub-band spectrum.

[0110] Optionally, in another embodiment of this application, a calculation of the power values ​​of various audio signals is provided, including:

[0111] For each target sample frequency band, the amplitude ratio of the target sample frequency band to base 10 is calculated by multiplying it by 20 to obtain the power value of the target sample frequency band.

[0112] The target sample frequency bands include a first sample sensitive frequency band, a second sample sensitive frequency band, a first sample non-sensitive frequency band, and a second sample non-sensitive frequency band. Therefore, the power value of the first sample sensitive frequency band is obtained by calculating the amplitude ratio of the first sample sensitive frequency band to the base-10 factor of 20. Similarly, the power value of the second sample sensitive frequency band is obtained by calculating the amplitude ratio of the first sample non-sensitive frequency band to the base-10 factor of 20.

[0113] Wherein, the amplitude ratio of the target sample frequency band is the ratio of the absolute value of the complex amplitude of the target sample frequency band to the reference reference. The reference parameter is the maximum value of the complex amplitude of all audio signals in the sample audio. Therefore, the formula for calculating the power value of the target sample frequency band can be specifically expressed as:

[0114]

[0115] Where |X(k)| is the complex amplitude corresponding to the k-th target sample frequency band mel_bin. reference is the reference benchmark value. In this embodiment, the complex amplitude of the largest target sample frequency band mel_bin in the entire audio data is selected as the reference benchmark.

[0116] Optionally, in order to facilitate the determination of the weights corresponding to the squared differences of the first sub-band spectrum, the second sub-band spectrum, the third sub-band spectrum, and the fourth sub-band spectrum, that is, to facilitate the determination of four types of weights, and to require that the weights of the squared differences of the spectrum of the same frequency band be different, four weight functions can be set to calculate the corresponding weights respectively.

[0117] Specifically, alternatively, in another embodiment of this application, one specific implementation of step S106 includes:

[0118] Based on the power values ​​of the first sample sensitive frequency band, the weights are calculated using the first weighting function to obtain the weights corresponding to the squared difference of the first sub-band spectrum. Based on the power values ​​of the second sample sensitive frequency band, the weights are calculated using the second weighting function to obtain the weights corresponding to the squared difference of the second sub-band spectrum.

[0119] Specifically, the power values ​​of each mel_bin belonging to the first sample sensitive frequency band are substituted into the first weighting function to calculate the weight corresponding to the squared difference of the spectrum of each mel_bin belonging to the first sample sensitive frequency band, which is the weight corresponding to the squared difference of the spectrum of each first sub-band. Similarly, the power values ​​of each mel_bin belonging to the second sample sensitive frequency band are substituted into the second weighting function to calculate the weight corresponding to the squared difference of the spectrum of each mel_bin belonging to the second sample sensitive frequency band, which is the weight corresponding to the squared difference of the spectrum of each second sub-band.

[0120] In other words, corresponding first and second weighting functions are set for the frequency bands above and below the first auditory masking threshold, respectively. The slope of the first weighting function is less than the slope of the second weighting function, and both are positively correlated functions. Therefore, the weight calculated using the first weighting function (i.e., the weight corresponding to the squared difference of the first sub-band spectrum) based on the power value of the first sample sensitive frequency band below the first auditory masking threshold is less than the weight calculated using the second weighting function (i.e., the weight corresponding to the squared difference of the second sub-band spectrum) based on the power value of the second sample sensitive frequency band above the first auditory masking threshold. Furthermore, this also satisfies the requirement that the larger the power value of each first and second sample sensitive frequency band, the larger the weight corresponding to its squared difference of spectrum.

[0121] Based on the power values ​​of the non-sensitive frequency bands of the first sample, the weights are calculated using the third weighting function to obtain the weights corresponding to the squared differences of the spectrum of the third sub-band. Based on the power values ​​of the non-sensitive frequency bands of the second sample, the weights are calculated using the fourth weighting function to obtain the weights corresponding to the squared differences of the spectrum of the fourth sub-band.

[0122] Similarly, both the third and fourth weighting functions are positive correlation functions, and the slope of the fourth weighting function is greater than that of the third weighting function. This ensures that the weight calculated by the third weighting function based on the power value of the first sample non-sensitive frequency band below the second auditory masking threshold, i.e., the weight corresponding to the squared difference of the third sub-band spectrum, is less than the weight calculated by the fourth weighting function based on the power value of the second sample non-sensitive frequency band above the second auditory masking threshold, i.e., the weight corresponding to the squared difference of the fourth sub-band spectrum.

[0123] Furthermore, the slope of the third weighting function is less than that of the first weighting function, and the slope of the fourth weighting function is less than that of the second weighting function. This ensures that the weights calculated using these four functions meet the requirement that the weight corresponding to the squared difference of the spectrum in the human ear-sensitive frequency band for the same power value is greater than the weight corresponding to the squared difference of the spectrum in the non-human-sensitive frequency band.

[0124] Optionally, another embodiment of this application provides a method for calculating the weights corresponding to the squared differences of the spectrum of the first sub-band based on the power values ​​of the first sample sensitive frequency band using a first weighting function, and the weights corresponding to the squared differences of the spectrum of the second sub-band based on the power values ​​of the second sample sensitive frequency band using a second weighting function, comprising:

[0125] The weights corresponding to the squared differences of the first sub-band spectrum are obtained by calculating twice the natural logarithm to the power of M using the first weighting function; where M is the power value of the non-sensitive frequency band of the first sample.

[0126] Where M is the power value P(x) of the first sample non-sensitive frequency band x.

[0127] In other words, for each mel_bin belonging to the non-sensitive frequency band of the first sample, the weight calculation function, i.e., the first weight function, is:

[0128]

[0129] The weights corresponding to the squared differences in the second sub-band spectrum are obtained by calculating twice the natural logarithm to the power of N using the second weighting function.

[0130] Where N is twice the power value P(x) of the second sample sensitive frequency band x.

[0131] In other words, for each mel_bin belonging to the sensitive frequency band of the second sample, the weight calculation function, i.e., the second weight function, is:

[0132]

[0133] Correspondingly, alternatively, another embodiment of this application provides a method for calculating the weights corresponding to the squared differences of the spectrum of the third sub-band based on the power values ​​of the non-sensitive frequency bands of the first sample using a third weighting function, and for calculating the weights corresponding to the squared differences of the spectrum of the fourth sub-band based on the power values ​​of the non-sensitive frequency bands of the second sample using a fourth weighting function, comprising:

[0134] The weights corresponding to the squared differences of the third subband spectrum are obtained by calculating the natural logarithm to the power of K using the third weighting function.

[0135] Where K is the power value P(x) of the first sample non-sensitive frequency band x.

[0136] In other words, for each mel_bin belonging to the non-sensitive frequency band of the first sample, the weight calculation function, i.e., the third weight function, is:

[0137]

[0138] The weights corresponding to the squared differences of the fourth sub-band spectrum are obtained by calculating the Lth power of the natural object using the fourth weighting function.

[0139] Where L is twice the power value P(x) of the second sample non-sensitive frequency band x.

[0140] In other words, for each mel_bin belonging to the non-sensitive frequency band of the second sample, the weight calculation function, i.e., the fourth weight function, is:

[0141]

[0142] It should be noted that in this embodiment, the dependent variable can vary significantly even when the independent variable changes little. Therefore, using an exponential function as the weighting function and setting different powers allows for greater differences in the weights of mel_bins with different power values, different frequency bands, and those above and below the absolute auditory masking threshold, thus more significantly reflecting their importance. Furthermore, it ensures that mel_bins with very low power values, equivalent to invalid audio, receive sufficiently small weights, thereby compressing them to zero and achieving adequate audio compression.

[0143] S107. Based on the squared difference of the spectrum of the first sub-band and its corresponding weight, the squared difference of the spectrum of the second sub-band and its corresponding weight, the squared difference of the spectrum of the third sub-band and its corresponding weight, and the squared difference of the spectrum of the fourth sub-band and its corresponding weight, the spectral Mel sub-band loss is obtained.

[0144] Specifically, the weights corresponding to the squared differences of the first sub-band spectrum are multiplied by the squared differences of the first sub-band spectrum and then weighted to obtain the first sub-band spectral loss. The weights corresponding to the squared differences of the second sub-band spectrum are multiplied by the squared differences of the second sub-band spectrum to obtain the second sub-band spectral loss. The weights corresponding to the squared differences of the third sub-band spectrum are multiplied by the squared differences of the third sub-band spectrum to obtain the third sub-band spectral loss. The weights corresponding to the squared differences of the fourth sub-band spectrum are multiplied by the squared differences of the fourth sub-band spectrum to obtain the fourth sub-band spectral loss. Furthermore, the spectral losses of the first, second, third, and fourth sub-bands are combined to obtain the Mel sub-band spectral loss. Alternatively, the Mel sub-band spectral loss can be obtained by calculating the mean of the spectral losses of the first, second, third, and fourth sub-bands.

[0145] S108. The initial audio compression model is optimized based on the spectral Mel subband loss to obtain the target audio compression model.

[0146] Specifically, the initial audio compression model is optimized using spectral Mel subband loss and other losses to obtain the optimized target audio compression model.

[0147] By weighting the spectral losses of each frequency band in the sample and predicted audio by calculating the weights of the frequency bands, important audio signals are enhanced while relatively unimportant audio signals are weakened. Therefore, using the weighted losses to calculate the spectral Mel-subband loss allows for greater focus on the losses of important frequency bands, thus reducing the focus on unimportant audio signals. Updating the parameters of the initial audio compression model using the current spectral Mel-subband loss ensures that the model prioritizes compression of important frequency bands, avoiding excessive compression of key audio signals and preventing the loss of valid information. Simultaneously, it allows for sufficient compression of relatively unimportant audio signals, maximizing their compression. Ultimately, this achieves adequate audio data compression without eliminating valid information.

[0148] Therefore, the logic of the method provided in this application embodiment can be specifically as follows: Figure 3 As shown, a Short Time Fourier Transform (STFT) is performed on the sample audio i and the predicted audio j. Then, according to the frequency bands sensitive to the human ear, the audio is divided into two parts: 2kHz-5kHz and <2kHz or >5kHz. Then, the mel_bin values ​​in each of the two parts are analyzed. i and mel_bin j Based on the set power threshold, i.e., the auditory masking threshold, the weights are calculated using the corresponding weighting function, and then each mel_bin is used. iand mel_bin j The weights of the spectral squared differences are used to weight the corresponding spectral squared differences, thus obtaining the weighted updated spectral squared differences (mel_bin). i 1-mel_bin j 1) 2 The weighted average of the two parts is used to calculate the spectral Mel subband loss (mel_loss), which is then used to update the parameters of the initial audio compression model.

[0149] This application provides a method for processing an audio compression model. An initial audio compression model compresses sample audio to obtain predicted audio. The sample audio and predicted audio are then divided into frequency bands: a sample sensitive frequency band, a sample non-sensitive frequency band, a predicted sensitive frequency band, and a predicted non-sensitive frequency band. This divides the two audio samples into two parts: one sensitive to human hearing and the other non-sensitive to human hearing. This allows for separate processing of these two parts, making the model more focused on audio signals within the sensitive frequency band—that is, focusing on more effective audio data that the human ear can easily and clearly perceive—avoiding over-compression and loss of effective information, while still ensuring sufficient compression of audio signals within the sensitive frequency band. Then, based on a first auditory masking threshold, a first and second auditory sensitive weights are determined for the sample sensitive frequency band and the predicted sensitive frequency band. Similarly, based on a second auditory masking threshold, a first and second non-auditory sensitive weights are determined for the sample non-sensitive frequency band and the predicted non-auditory non-sensitive frequency band. This allows for the allocation of greater weights to more effective audio signals and smaller weights to relatively less important audio signals through the set auditory masking thresholds. Then, based on the first auditory masking threshold, the sample sensitive frequency band and the predicted sensitive frequency band are divided into the first sample sensitive frequency band and the second sample sensitive frequency band, as well as the first predicted sensitive frequency band and the second predicted sensitive frequency band. Similarly, based on the second auditory masking threshold, the sample insensitive frequency band and the predicted insensitive frequency band are divided into the first sample insensitive frequency band and the second sample insensitive frequency band, as well as the first predicted insensitive frequency band and the second predicted insensitive frequency band. This further divides the auditory masked and unmasked frequency bands to allow for targeted analysis of their losses. Then, the squared difference of the first sub-band spectrum between the first sample sensitive frequency band and the first predicted sensitive frequency band, the squared difference of the second sub-band spectrum between the second sample sensitive frequency band and the second predicted sensitive frequency band, the squared difference of the third sub-band spectrum between the first sample insensitive frequency band and the first predicted insensitive frequency band, and the squared difference of the fourth sub-band spectrum corresponding to the second sample insensitive frequency band and the second predicted insensitive frequency band are calculated respectively. This determines the initial spectral error of each frequency band, which is used for subsequent calculation of the loss of each frequency band. Next, the weights corresponding to the squared differences of the first, second, third, and fourth sub-band spectra are determined. Based on the squared differences of the first, second, third, and fourth sub-band spectra and their corresponding weights, the Mel sub-band loss is obtained. By calculating the corresponding weights for each frequency band, the spectrum is weighted, thus giving more attention to the data in important frequency bands while avoiding overemphasizing the data in relatively unimportant frequency bands. This effectively avoids the loss of useful information and allows for sufficient compression of unimportant data.Finally, the initial audio compression model is optimized based on the spectral Mel subband loss to obtain the target audio compression model. This target audio compression model can avoid the loss of effective information and fully compress the audio to achieve the best compression ratio.

[0150] Another embodiment of this application provides a processing system for an audio compression model, such as... Figure 4 As shown, it includes:

[0151] The compression processing unit 401 is used to compress the sample audio using an initial audio compression model to obtain the predicted audio.

[0152] The audio segmentation unit 402 is used to segment the sample audio and the predicted audio into frequency bands respectively, to obtain the sample sensitive frequency band, the sample non-sensitive frequency band, the predicted sensitive frequency band, and the predicted non-sensitive frequency band.

[0153] The sensitive frequency band segmentation unit 403 is used to segment the sample sensitive frequency band and the predicted sensitive frequency band based on the first auditory masking threshold, so as to obtain the first sample sensitive frequency band and the second sample sensitive frequency band, as well as the first predicted sensitive frequency band and the second predicted sensitive frequency band.

[0154] The non-sensitive frequency band segmentation unit 404 is used to segment the sample non-sensitive frequency band and the predicted non-sensitive frequency band based on the second auditory masking threshold, respectively, to obtain the first sample non-sensitive frequency band and the second sample non-sensitive frequency band, as well as the first predicted non-sensitive frequency band and the second predicted non-sensitive frequency band.

[0155] The deviation calculation unit 405 is used to calculate the squared difference of the first sub-band spectrum between the first sample sensitive frequency band and the first predicted sensitive frequency band, the squared difference of the second sub-band spectrum between the second sample sensitive frequency band and the second predicted sensitive frequency band, the squared difference of the third sub-band spectrum between the first sample non-sensitive frequency band and the first predicted non-sensitive frequency band, and the squared difference of the fourth sub-band spectrum between the second sample non-sensitive frequency band and the second predicted non-sensitive frequency band.

[0156] The weight analysis unit 406 is used to determine the weights corresponding to the squared differences of the first sub-band spectrum, the second sub-band spectrum, the third sub-band spectrum, and the fourth sub-band spectrum.

[0157] The sample audio weighting unit 407 is used to obtain the spectral Mel subband loss based on the squared difference of the first subband spectrum and its corresponding weight, the squared difference of the second subband spectrum and its corresponding weight, the squared difference of the third subband spectrum and its corresponding weight, and the squared difference of the fourth subband spectrum and its corresponding weight.

[0158] Model optimization unit 408 is used to optimize the initial audio compression model based on the spectral Mel subband loss to obtain the target audio compression model.

[0159] Optionally, in the audio compression model processing system provided in another embodiment of this application, the audio partitioning unit includes:

[0160] The transform unit is used to perform short-time Fourier transforms on the sample audio and the predicted audio, respectively.

[0161] The frequency band division unit is used to divide the Mel subband frequency band of the transformed sample audio into sample-sensitive and sample-insensitive frequency bands according to the human ear's sensitive frequency band, and to divide the Mel subband frequency band of the transformed predicted audio into prediction-sensitive and prediction-insensitive frequency bands according to the human ear's sensitive frequency band. The human ear-sensitive frequency band is from 2 kHz to 5 kHz.

[0162] Optionally, in the audio compression model processing system provided in another embodiment of this application, the weight analysis unit includes:

[0163] The first weight calculation unit is used to calculate the weights based on the power values ​​of the first sample sensitive frequency band using a first weight function, obtaining the weights corresponding to the squared differences of the first sub-band spectrum; and to calculate the weights based on the power values ​​of the second sample sensitive frequency band using a second weight function, obtaining the weights corresponding to the squared differences of the second sub-band spectrum. The slope of the first weight function is less than the slope of the second weight function.

[0164] The second weighting calculation unit is used to calculate the weights based on the power values ​​of the non-sensitive frequency bands of the first sample using a third weighting function, obtaining the weights corresponding to the squared differences of the spectrum in the third sub-band; and to calculate the weights based on the power values ​​of the non-sensitive frequency bands of the second sample using a fourth weighting function, obtaining the weights corresponding to the squared differences of the spectrum in the fourth sub-band. The slope of the third weighting function is less than the slope of the first weighting function. The slope of the fourth weighting function is less than the slope of the second weighting function but greater than the slope of the third weighting function.

[0165] Optionally, in the audio compression model processing system provided in another embodiment of this application, the first weight calculation unit includes:

[0166] The first calculation unit is used to calculate twice the natural logarithm raised to the power of M using a first weighting function to obtain the weight corresponding to the squared difference of the first sub-band spectrum. Here, M is the power value of the non-sensitive frequency band of the first sample.

[0167] The second calculation unit is used to calculate the weight corresponding to the squared difference of the second sub-band spectrum by power N, which is twice the natural logarithm, through the second weighting function. Here, N is twice the power value of the second sample sensitive frequency band.

[0168] Optionally, in the audio compression model processing system provided in another embodiment of this application, the second weight calculation unit includes:

[0169] The third calculation unit is used to calculate the natural logarithm to the power of K using the third weighting function, thereby obtaining the weight corresponding to the squared difference of the third sub-band spectrum. Here, K is the power value of the non-sensitive frequency band of the first sample.

[0170] The fourth calculation unit is used to calculate the L-th power of the natural object using the fourth weighting function, thereby obtaining the weight corresponding to the squared difference of the fourth sub-band spectrum. Here, L is twice the power value of the second sample's non-sensitive frequency band.

[0171] Optionally, in the audio compression model processing system provided in another embodiment of this application, the following further includes:

[0172] The power value calculation unit is used to calculate the power values ​​of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band based on the complex amplitudes of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band, and the second sample non-sensitive frequency band, respectively.

[0173] Optionally, in the audio compression model processing system provided in another embodiment of this application, the power value calculation unit includes:

[0174] The power value calculation subunit is used to calculate the amplitude ratio of the target sample frequency band to a factor of twenty (base 10) for each target sample frequency band, thus obtaining the power value of the target sample frequency band. The target sample frequency bands include a first sample sensitive frequency band, a second sample sensitive frequency band, a first sample non-sensitive frequency band, and a second sample non-sensitive frequency band. The amplitude ratio of the target sample frequency band is the ratio of the absolute value of the complex amplitude of the target sample frequency band to a reference reference. The parameter reference is the maximum value of the complex amplitude among all Mel subbands in the sample audio.

[0175] It should be noted that the specific working process of each unit provided in the above embodiments of this application can be referred to the implementation process of the corresponding steps in the above method embodiments, and will not be repeated here.

[0176] Another embodiment of this application provides a computer device, such as... Figure 5 As shown, it includes a memory 501 and a processor 502.

[0177] Memory 501 is used to store computer programs.

[0178] The processor 502 is used to execute a computer program to implement the processing method of the audio compression model provided in any of the above embodiments.

[0179] Another embodiment of this application provides a computer program product including executable computer instructions. When executed by a processor (e.g., the processor 502 described above), the computer instructions are used to implement a processing method for an audio compression model as provided in any of the above embodiments.

[0180] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A processing method of an audio compression model, characterized by, The method comprises the following steps: performing compression processing on the sample audio through an initial audio compression model to obtain predicted audio; performing frequency band division on the sample audio and the predicted audio respectively to obtain sample sensitive frequency bands, sample non-sensitive frequency bands, predicted sensitive frequency bands and predicted non-sensitive frequency bands; dividing the sample sensitive frequency bands and the predicted sensitive frequency bands based on a first auditory masking threshold to obtain first sample sensitive frequency bands and second sample sensitive frequency bands and first predicted sensitive frequency bands and second predicted sensitive frequency bands; dividing the sample non-sensitive frequency bands and the predicted non-sensitive frequency bands based on a second auditory masking threshold to obtain first sample non-sensitive frequency bands and second sample non-sensitive frequency bands and first predicted non-sensitive frequency bands and second predicted non-sensitive frequency bands; calculating first sub-band spectral square difference values of the first sample sensitive frequency bands and the first predicted sensitive frequency bands, second sub-band spectral square difference values of the second sample sensitive frequency bands and the second predicted sensitive frequency bands, third sub-band spectral square difference values of the first sample non-sensitive frequency bands and the first predicted non-sensitive frequency bands, and fourth sub-band spectral square difference values corresponding to the second sample non-sensitive frequency bands and the second predicted non-sensitive frequency bands; determining weights corresponding to the first sub-band spectral square difference values, weights corresponding to the second sub-band spectral square difference values, weights corresponding to the third sub-band spectral square difference values, and weights corresponding to the fourth sub-band spectral square difference values; obtaining spectral mel sub-band losses based on the first sub-band spectral square difference values and the weights corresponding thereto, the second sub-band spectral square difference values and the weights corresponding thereto, the third sub-band spectral square difference values and the weights corresponding thereto, and the fourth sub-band spectral square difference values and the weights corresponding thereto; optimizing the initial audio compression model based on the spectral mel sub-band losses to obtain a target audio compression model.

2. The method of claim 1, wherein, The method comprises the following steps: performing short-time Fourier transform on the sample audio and the predicted audio respectively; dividing mel sub-band frequency bands of the sample audio after transformation into sample sensitive frequency bands and sample non-sensitive frequency bands according to human ear sensitive frequency bands, and dividing mel sub-band frequency bands of the predicted audio after transformation into predicted sensitive frequency bands and predicted non-sensitive frequency bands according to human ear sensitive frequency bands; wherein the human ear sensitive frequency bands are 2-5 kHz.

3. The method of claim 1, wherein, The method comprises the following steps: based on power values of the first sample sensitive frequency bands, a first weight function is used to calculate weights to obtain weights corresponding to the first sub-band spectral square difference values, and based on power values of the second sample sensitive frequency bands, a second weight function is used to calculate weights to obtain weights corresponding to the second sub-band spectral square difference values; wherein a slope of the first weight function is smaller than a slope of the second weight function. The third weight function is used to calculate the weight based on the power value of the first sample non-sensitive frequency band, and the weight corresponding to the third sub-band spectral square difference value is obtained, and the fourth weight function is used to calculate the weight based on the power value of the second sample non-sensitive frequency band, and the weight corresponding to the fourth sub-band spectral square difference value is obtained; wherein the slope of the third weight function is less than the slope of the first weight function; the slope of the fourth weight function is less than the slope of the second weight function, and greater than the slope of the third weight function.

4. The method of claim 3, wherein, The first weight function is used to calculate the weight based on the power value of the first sample sensitive frequency band, and the weight corresponding to the first sub-band spectral square difference value is obtained, and the second weight function is used to calculate the weight based on the power value of the second sample sensitive frequency band, and the weight corresponding to the second sub-band spectral square difference value is obtained, including: The first weight function is used to calculate the weight corresponding to the first sub-band spectral square difference value by calculating the Mth power of twice the natural logarithm; wherein M is the power value of the first sample non-sensitive frequency band; The second weight function is used to calculate the weight corresponding to the second sub-band spectral square difference value by calculating the Nth power of twice the natural logarithm; wherein N is twice the power value of the second sample sensitive frequency band.

5. The method of claim 4, wherein, The third weight function is used to calculate the weight based on the power value of the first sample non-sensitive frequency band, and the weight corresponding to the third sub-band spectral square difference value is obtained, and the fourth weight function is used to calculate the weight based on the power value of the second sample non-sensitive frequency band, and the weight corresponding to the fourth sub-band spectral square difference value is obtained, including: The third weight function is used to calculate the weight corresponding to the third sub-band spectral square difference value by calculating the Kth power of the natural logarithm; wherein K is the power value of the first sample non-sensitive frequency band; The fourth weight function is used to calculate the weight corresponding to the fourth sub-band spectral square difference value by calculating the Lth power of the natural object; wherein L is twice the power value of the second sample non-sensitive frequency band.

6. The method of claim 1, wherein, Further comprising: The power values of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band and the second sample non-sensitive frequency band are calculated according to the complex amplitudes of the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band and the second sample non-sensitive frequency band, respectively.

7. The method of claim 6, wherein, The power values of each audio signal in the sample sensitive frequency band, the sample non-sensitive frequency band, the predicted sensitive frequency band and the predicted non-sensitive frequency band are calculated according to the complex amplitudes of each audio signal in the sample sensitive frequency band, the sample non-sensitive frequency band, the predicted sensitive frequency band and the predicted non-sensitive frequency band, respectively. The amplitude ratio of the target sample frequency band is calculated, and the power value of the target sample frequency band is obtained; wherein the target sample frequency band includes the first sample sensitive frequency band, the second sample sensitive frequency band, the first sample non-sensitive frequency band and the second sample non-sensitive frequency band; the amplitude ratio of the target sample frequency band is the ratio of the absolute value of the complex amplitude of the target sample frequency band to the reference benchmark; the parameter benchmark is the maximum value in the complex amplitude of all mel subband frequency bands in the sample audio.

8. A processing system of an audio compression model, characterized by, Comprise: A compression processing unit for compressing the sample audio by an initial audio compression model to obtain a predicted audio; An audio division unit for dividing the sample audio and the predicted audio into frequency bands respectively to obtain a sample sensitive frequency band, a sample non-sensitive frequency band, a predicted sensitive frequency band and a predicted non-sensitive frequency band; A sensitive frequency band division unit for dividing the sample sensitive frequency band and the predicted sensitive frequency band based on a first auditory masking threshold to obtain a first sample sensitive frequency band and a second sample sensitive frequency band and a first predicted sensitive frequency band and a second predicted sensitive frequency band; A non-sensitive frequency band division unit for dividing the sample non-sensitive frequency band and the predicted non-sensitive frequency band based on a second auditory masking threshold to obtain a first sample non-sensitive frequency band and a second sample non-sensitive frequency band and a first predicted non-sensitive frequency band and a second predicted non-sensitive frequency band; A bias calculation unit for calculating a first subband frequency spectrum square difference value of the first sample sensitive frequency band and the first predicted sensitive frequency band, a second subband frequency spectrum square difference value of the second sample sensitive frequency band and the second predicted sensitive frequency band, a third subband frequency spectrum square difference value of the first sample non-sensitive frequency band and the first predicted non-sensitive frequency band, and a fourth subband frequency spectrum square difference value of the second sample non-sensitive frequency band and the second predicted non-sensitive frequency band; A weight analysis unit for determining the weight corresponding to the first subband frequency spectrum square difference value, the weight corresponding to the second subband frequency spectrum square difference value, the weight corresponding to the third subband frequency spectrum square difference value, and the weight corresponding to the fourth subband frequency spectrum square difference value; A sample audio weighting unit for obtaining a spectral mel subband loss based on the first subband frequency spectrum square difference value and its corresponding weight, the second subband frequency spectrum square difference value and its corresponding weight, the third subband frequency spectrum square difference value and its corresponding weight, and the fourth subband frequency spectrum square difference value and its corresponding weight. A model optimization unit for optimizing the initial audio compression model based on the spectral mel subband loss to obtain a target audio compression model.

9. A computer device, comprising: Comprise a memory and a processor; The memory is used to store a computer program; The processor is used to execute the computer program to realize the processing method of the audio compression model according to any one of claims 1 to 7.

10. A computer program product, characterised in that, Comprise executable computer instructions; The computer instructions are executed by the processor to realize the processing method of the audio compression model according to any one of claims 1 to 7.

Images