Audio signal processing method and apparatus, storage medium, and computer program product

Abstract

An audio signal processing method and apparatus (1100), a storage medium, and a computer program product, relating to the field of audio encoding and decoding. According to the characteristics of an audio signal, an optimal sub-band division mode is selected from a plurality of sub-band division modes, that is, the sub-band division mode has the characteristic of signal self-adaptation, and can adapt to an encoding code rate of the audio signal, thereby improving the anti-interference capability. Division is first performed on an audio signal according to a plurality of sub-band division modes, respectively, and then a total scale value corresponding to each sub-band division mode is determined on the basis of a spectrum value of the audio signal in each sub-band obtained by the division, the bandwidth of each sub-band, and an encoding code rate of the audio signal, and an optimal target sub-band division mode is selected on the basis of the total scale value to obtain an optimal sub-band set. Subsequently, spectral envelope shaping is performed according to a scale factor of each sub-band in the optimal sub-band set, thereby improving the encoding effect and the compression efficiency.

Description

Audio signal processing method, device, storage medium and computer program product

[0001] This application claims priority to Chinese Patent Application No. 202210894324.9, filed on July 27, 2022, entitled “Audio Signal Processing Method, Apparatus, Storage Medium, and Computer Program Product,” the entire contents of which are incorporated herein by reference. This application also claims priority to Chinese Patent Application No. 202211139940.X, filed on September 19, 2022, entitled “Audio Signal Processing Method, Apparatus, Storage Medium, and Computer Program Product,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of audio coding and decoding, and in particular to a method, device, storage medium and computer program product for processing audio signals. Background Art

[0003] As people's quality of life improves, the demand for high-quality audio continues to grow. To better transmit audio signals using limited bandwidth, the audio signal is typically compressed at the encoder to generate a bitstream, which is then transmitted to the decoder. The decoder decodes the received bitstream to reconstruct the audio signal for playback. However, the compression process can affect the audio quality. Therefore, improving audio compression efficiency while maintaining audio quality has become a pressing technical challenge.

[0004] Summary of the Invention

[0005] This application provides an audio signal processing method, device, storage medium, and computer program product, which can improve encoding effect and compression efficiency. The technical solution is as follows:

[0006] In a first aspect, a method for processing an audio signal is provided, the method comprising:

[0007] The audio signal is sub-divided according to multiple sub-band division modes and cutoff sub-bands corresponding to the multiple sub-band division modes to obtain multiple candidate sub-band sets. The multiple candidate sub-band sets correspond one-to-one to the multiple sub-band division modes, and each candidate sub-band set includes multiple sub-bands. A total scale value of each candidate sub-band set is determined based on a spectral value of the audio signal in the sub-bands included in each candidate sub-band set, a coding bit rate of the audio signal, and a sub-band bandwidth of the sub-bands included in each candidate sub-band set. According to the total scale value of each candidate sub-band set, a candidate sub-band set is selected from the multiple candidate sub-band sets as a target sub-band set. Each sub-band included in the target sub-band set has a scale factor, and the scale factor is used to shape the spectral envelope of the audio signal.

[0008] In this application, based on the characteristics of the audio signal, the optimal sub-band division method is selected from multiple sub-band division methods. That is, the sub-band division method has the characteristics of signal adaptation and can adapt to the encoding bit rate of the audio signal, thereby improving anti-interference capabilities. Specifically, the audio signal is first divided according to multiple sub-band division methods. Then, based on the spectral values ​​of the audio signal in each divided sub-band, the bandwidth of each sub-band, and the encoding bit rate of the audio signal, the total scale value corresponding to each sub-band division method is determined. Based on the total scale value, the optimal target sub-band division method is selected, that is, the optimal sub-band set is obtained. Subsequent spectrum envelope shaping according to the scaling factors of each sub-band in the optimal sub-band set can improve the encoding effect and compression efficiency.

[0009] Optionally, selecting a candidate subband set as a target subband set from the multiple candidate subband sets according to the total scale value of each candidate subband set includes:

[0010] A candidate subband set with the smallest total scale value among the multiple candidate subband sets is determined as the target subband set.

[0011] Optionally, determining the total scale value of each candidate subband set based on a spectrum value of the audio signal in a subband included in each candidate subband set, a coding bit rate of the audio signal, and a subband bandwidth of the subband included in each candidate subband set includes:

[0012] determining, for a first candidate subband set among the multiple candidate subband sets, a scale factor for each subband included in the first candidate subband set based on spectral values ​​of the audio signal in each subband included in the first candidate subband set, where the first candidate subband set is any candidate subband set among the multiple candidate subband sets;

[0013] An overall scaling value of the first candidate subband set is determined based on a coding rate of the audio signal and a scaling factor and a subband bandwidth of each subband included in the first candidate subband set.

[0014] Optionally, determining, based on a spectrum value of the audio signal in each subband included in the first candidate subband set, a scale factor for each subband included in the first candidate subband set includes:

[0015] For a first subband included in the first candidate subband set, obtaining a maximum absolute value of all spectral values ​​of the audio signal in the first subband, where the first subband is any subband in the first candidate subband set;

[0016] Based on the maximum value, a scale factor for the first subband is determined.

[0017] Optionally, the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold;

[0018] The determining, based on the encoding bit rate of the audio signal and the scale factor and subband bandwidth of each subband included in the first candidate subband set, a total scale value of the first candidate subband set includes:

[0019] determining an energy smoothing reference value based on the encoding bit rate of the audio signal and a second bit rate threshold;

[0020] determining a total energy value of each subband included in the first candidate subband set based on the energy smoothing reference value, a scale factor and a subband bandwidth of each subband included in the first candidate subband set;

[0021] Total energy values ​​of the subbands included in the first candidate subband set are added together to obtain a total scale value of the first candidate subband set.

[0022] Optionally, determining the total energy value of each subband included in the first candidate subband set based on the energy smoothing reference value, the scale factor and the subband bandwidth of each subband included in the first candidate subband set includes:

[0023] For a first subband included in the first candidate subband set, determining a maximum value between a scale factor of the first subband and the energy smoothing reference value as a reference scale value of the first subband, where the first subband is any subband in the first candidate subband set;

[0024] A product of the reference scale value of the first sub-band and the sub-band bandwidth of the first sub-band is determined as the total energy value of the first sub-band.

[0025] Optionally, the encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold;

[0026] The determining, based on the encoding bit rate of the audio signal and the scale factor and subband bandwidth of each subband included in the first candidate subband set, a total scale value of the first candidate subband set includes:

[0027] determining an energy smoothing reference value based on the encoding bit rate of the audio signal and a second bit rate threshold;

[0028] determining, based on the energy smoothing reference value and the scale factor of each subband included in the first candidate subband set, a scale difference value for each subband included in the first candidate subband set, wherein the scale difference value represents a difference between the scale factor of the corresponding subband and the scale factor of an adjacent subband of the corresponding subband;

[0029] An overall scaling value of the first candidate subband set is determined based on the scaling difference value and the subband bandwidth of each subband included in the first candidate subband set.

[0030] Optionally, determining the scale difference value of each subband included in the first candidate subband set based on the energy smoothing reference value and the scale factor of each subband included in the first candidate subband set includes:

[0031] For a first subband included in the first candidate subband set, determining a first smoothing value, a second smoothing value, and a third smoothing value of the first subband based on the energy smoothing reference value, a scale factor of the first subband, and scale factors of adjacent subbands of the first subband, where the first subband is any subband in the first candidate subband set;

[0032] A scaled difference value of the first subband is determined based on the first smoothed value, the second smoothed value, and the third smoothed value of the first subband.

[0033] Optionally, determining the first smoothing value, the second smoothing value, and the third smoothing value of the first subband based on the energy smoothing reference value, the scale factor of the first subband, and the scale factors of adjacent subbands of the first subband includes:

[0034] If the first subband is the first subband in the first candidate subband set, determining the maximum value of the scale factor of the first subband and the energy smoothing reference value as the first smoothing value of the first subband; if the first subband is not the first subband in the first candidate subband set, determining the maximum value of the scale factor of the previous adjacent subband of the first subband and the energy smoothing reference value as the first smoothing value of the first subband;

[0035] determining a maximum value of the scaling factor of the first sub-band and the energy smoothing reference value as a second smoothing value of the first sub-band;

[0036] If the first subband is the last subband in the first candidate subband set, the maximum value of the scale factor of the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband; if the first subband is not the last subband in the first candidate subband set, the maximum value of the scale factor of the next adjacent subband of the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband.

[0037] Optionally, determining the scale difference value of the first subband based on the first smoothed value, the second smoothed value, and the third smoothed value of the first subband includes:

[0038] determining, for a first subband included in the first candidate subband set, a first difference value and a second difference value of the first subband, where the first difference value refers to an absolute value of a difference between a first smoothed value and a second smoothed value of the first subband, and the second difference value refers to an absolute value of a difference between a second smoothed value and a third smoothed value of the first subband, wherein the first subband is any subband in the first candidate subband set;

[0039] A scaled disparity value for the first subband is determined based on the first disparity value and the second disparity value of the first subband.

[0040] Optionally, determining the total scale value of the first candidate subband set based on the scale difference value and the subband bandwidth of each subband included in the first candidate subband set includes:

[0041] determining, based on the number of subbands included in the first candidate subband set and the subband bandwidth of each subband, a smoothing weight coefficient for each subband included in the first candidate subband set;

[0042] adding the smoothing weight coefficients of the subbands included in the first candidate subband set to obtain a total smoothing weight coefficient of the first candidate subband set;

[0043] multiplying the scale difference value of each subband included in the first candidate subband set by a smoothing weight coefficient to obtain a weighted scale difference value of each subband included in the first candidate subband set;

[0044] adding weighted scale difference values ​​of the subbands included in the first candidate subband set to obtain a summed scale value of the first candidate subband set;

[0045] The summed scale value of the first candidate subband set is divided by the total smoothing weight coefficient to obtain a total scale value of the first candidate subband set.

[0046] Optionally, the method further includes:

[0047] If the encoding bit rate of the audio signal is less than a first bit rate threshold, performing bandwidth detection on the spectrum of the audio signal to obtain a cutoff frequency of the audio signal;

[0048] Based on the cutoff frequency, cutoff sub-bands corresponding to the multiple sub-band division modes are determined.

[0049] Optionally, the method further includes:

[0050] If the encoding bit rate of the audio signal is not less than the first bit rate threshold, determining the last subband indicated by each subband division method among the multiple subband division methods as the various subbands. Optionally, the method further includes:

[0051] Performing feature analysis on the frequency spectrum of the audio signal to obtain a feature analysis result;

[0052] The multiple sub-band division modes are determined from multiple candidate sub-band division modes based on the feature analysis result and the encoding bit rate of the audio signal.

[0053] Optionally, the feature analysis result includes a subjective signal flag or an objective signal flag, the subjective signal flag indicates that the energy concentration of the audio signal is not greater than a concentration threshold, and the objective signal flag indicates that the energy concentration of the audio signal is greater than the concentration threshold.

[0054] Optionally, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0055] The determining, based on the feature analysis result and the encoding bit rate of the audio signal, the multiple sub-band division modes from multiple candidate sub-band division modes includes:

[0056] If the encoding bit rate of the audio signal is less than a first bit rate threshold, and the feature analysis result includes the subjective signal flag, determining a first group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods;

[0057] The first group of subbands is divided as follows: { {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142, 160,178,196,217,238,259,280,480}, {0,1,2,3,5,7,9,12,15,18,22,26,30,35,41,48,56,65,74,84,94,106,118,134,150, 166,184,202,220,240,260,280,480}, {0,1,2,3,4,5,7,9,11,14,17,21,25,29,34,40,46,52,60,68,76,86,98,110,126,144, 162,180,200,224,250,280,480}, {0,2,4,6,8,12,16,21,26,31,36,41,46,51,56,61,66,71,77,83,89,95,103,111,121, 131,147,163,179,203,240,280,480}, {0,1,2,3,5,7,9,12,15,19,23,27,32,37,43,49,57,66,76,86,98,110,125,140,158, 176,194,216,238,264,290,320,480}, {0,1,2,3,5,7,10,13,17,21,25,30,35,41,47,54,62,70,80,90,102,114,130,146,162, 180,198,218,240,264,290,320,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,66,76,88,100,112,128,144,160,182, 204,226,256,286,316,352,400,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,68,78,90,102,116,132,148,166,186, 208,234,262,292,324,360,400,480}}.

[0058] Optionally, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0059] The determining, based on the feature analysis result and the bit rate of the audio signal, the multiple sub-band division modes from multiple candidate sub-band division modes includes:

[0060] If the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the feature analysis result includes the objective signal flag, determining a second group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods;

[0061] The second group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,26,30,35,40,45,50,57,64,73,82,92,102, 112,124,136,148,160,480}, {0,1,2,3,4,5,7,9,11,13,15,18,21,24,28,33,38,44,50,57,64,73,82,93,104,116, 128,140,155,170,185,200,480}, {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142, 160,178,196,217,238,259,280,480}, {0,1,2,4,6,10,14,18,22,26,30,34,42,50,58,66,74,84,96,108,120,136,152,168, 192,216,240,272,304,336,376,424,480}, {0,1,2,4,6,10,14,18,26,34,42,50,62,74,86,98,112,128,144,160,176,196,216,236, 256,280,304,328,352,384,416,448,480}, {0,80,92,104,112,120,128,136,144,148,152,156,160,164,168,172,176,180,184, 188,192,196,200,208,216,224,232,240,248,256,268,280,480}, {0,200,212,224,232,240,248,256,264,268,272,276,280,284,288,292,296,300,304, 308,312,316,320,328,336,344,352,360,368,376,388,400,480}, {0,320,332,344,356,364,372,380,384,388,392,396,400,404,408,412,416,420,424, 428,432,436,440,444,448,452,456,460,464,468,472,476,480}}.

[0062] Optionally, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0063] The determining, based on the feature analysis result and the bit rate of the audio signal, the multiple sub-band division modes from multiple candidate sub-band division modes includes:

[0064] If the encoding bit rate of the audio signal is less than a first bit rate threshold, and the feature analysis result includes the subjective signal flag, determining a third group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods;

[0065] The third group of subbands is divided as follows: {{0,1,2,3,4,5,6,7,8,9,10,12,14,16,19,22,26,30,35,39,44,50,56,63,71,80,89, 98,108,119,129,140,240}, {0,1,2,3,4,5,6,7,8,9,11,13,15,17,20,24,28,32,37,42,47,53,59,67,75,83,92, 101,110,120,130,140,240}, {0,1,2,3,4,5,6,7,8,9,10,11,12,14,17,20,23,26,30,34,38,43,49,55,63,72,81,90, 100,112,125,140,240}, {0,1,2,3,4,6,8,10,13,15,18,20,23,25,28,30,33,35,38,41,44,47,51,55,60,65,73, 81,89,101,120,140,240}, {0,1,2,3,4,5,6,7,8,10,12,14,17,20,23,27,31,35,40,45,51,57,65,73,81,90,99, 109,120,132,145,160,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,33,38,44,50,56,64,72,80,91,102,113, 128,143,158,176,200,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,34,39,45,51,58,66,74,83,93,104,117, 131,146,162,180,200,240}}.

[0066] Optionally, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0067] The determining, based on the feature analysis result and the bit rate of the audio signal, the multiple sub-band division modes from multiple candidate sub-band division modes includes:

[0068] If the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the feature analysis result includes the objective signal flag, determining a fourth group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods;

[0069] The fourth group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,30, 32,34,37,40,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20,22,24,26,28,30,32,34,36,38, 41,44,47,50,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,17,19,21,24,27,30,34,38,42,48,54,60,68, 76,84,94,106,120}, {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,25,28,32,36,40,44,49,54,59,64,70,76,82, 88,96,104,112,120}, {0,20,23,26,28,30,32,34,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,52,54, 56,58,60,62,64,67,70,120}, ,98,99,100,101,102,103,104,105,106,107,108,109, 110,111,112,113,114,115,116,117,118,119,120}}.

[0070] Optionally, the audio signal is a two-channel signal;

[0071] The method further comprises:

[0072] Determining a first total scaling value based on a scaling factor and a subband bandwidth of each subband included in the target subband set;

[0073] Performing an addition-subtraction stereo transformation on the spectrum of the two-channel signal to obtain a transformed spectrum of the two-channel signal;

[0074] determining a scale factor after transformation of each subband in the target subband set based on a spectrum value of the transformed binaural signal in each subband included in the target subband set;

[0075] determining a second total scaling value based on the transformed scaling factors and subband bandwidths of the subbands included in the target subband set;

[0076] If the first total scale value is not greater than the second total scale value, the two-channel signal is determined as a signal to be encoded.

[0077] Optionally, the method further includes:

[0078] If the first total scale value is greater than the second total scale value, and the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold, the transformed binaural signal is determined as the signal to be encoded.

[0079] Optionally, the scale factor includes a left channel scale factor and a right channel scale factor;

[0080] The method further comprises:

[0081] If the first total scale value is greater than the second total scale value, the encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold, determining a left-right scale factor difference value for each subband included in the target subband set based on the left channel scale factor and the right channel scale factor of each subband included in the target subband set;

[0082] Determining a subband center frequency of each subband included in the target subband set based on an initial frequency point and a cutoff frequency point of each subband included in the target subband set;

[0083] If there is at least one subband in the target subband set whose left and right scale factor difference value is greater than a difference threshold and whose subband center frequency is within a first range, the binaural signal is determined as a signal to be encoded.

[0084] Optionally, the method further includes:

[0085] If the at least one subband does not exist in the target subband set, the transformed binaural signal is determined as the signal to be encoded.

[0086] In a second aspect, an audio signal processing device is provided, wherein the audio signal processing device has the function of implementing the audio signal processing method described in the first aspect. The audio signal processing device includes one or more modules configured to implement the audio signal processing method described in the first aspect.

[0087] In a third aspect, a device for processing audio signals is provided. The device includes a processor and a memory, wherein the memory is configured to store a program for executing the audio signal processing method provided in the first aspect, as well as data used to implement the audio signal processing method provided in the first aspect. The processor is configured to execute the program stored in the memory. The device may also include a communication bus for establishing a connection between the processor and the memory.

[0088] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the computer-readable storage medium is run on a computer, the computer executes the audio signal processing method described in the first aspect.

[0089] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the audio signal processing method described in the first aspect.

[0090] The technical effects obtained in the above-mentioned second, third, fourth and fifth aspects are similar to those obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] FIG1 is a schematic diagram of a Bluetooth interconnection scenario provided by an embodiment of the present application;

[0092] FIG2 is a system framework diagram of the audio signal processing method provided in an embodiment of the present application;

[0093] FIG3 is a diagram of an overall framework of an audio codec provided in an embodiment of the present application;

[0094] FIG4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0095] FIG5 is a flowchart of a method for processing an audio signal provided in an embodiment of the present application;

[0096] FIG6 is a diagram showing the relationship between the sub-bands and the initial frequency points indicated by the first group of sub-band division methods provided in an embodiment of the present application;

[0097] FIG7 is a diagram showing the relationship between the sub-bands and the initial frequency points indicated by the second group of sub-band division methods provided in an embodiment of the present application;

[0098] FIG8 is a diagram showing the relationship between the sub-bands and the initial frequency points indicated by the third group of sub-band division methods provided in an embodiment of the present application;

[0099] FIG9 is a diagram showing the relationship between the sub-bands and the initial frequency points indicated by the fourth group of sub-band division methods provided in an embodiment of the present application;

[0100] FIG10 is a flow chart of a method for determining whether MS conversion is profitable provided in an embodiment of the present application;

[0101] FIG11 is a schematic structural diagram of an audio signal processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0102] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0103] First, the implementation environment and background knowledge involved in the embodiments of this application are introduced.

[0104] With the widespread popularity and use of wireless Bluetooth devices such as true wireless stereo (TWS) headphones, smart speakers, and smart watches in people's daily lives, people's demand for high-quality audio playback experience in various scenarios has become increasingly urgent, especially in environments such as subways, airports, and train stations where Bluetooth signals are susceptible to interference. In Bluetooth interconnection scenarios, due to the limitations of the Bluetooth channel connecting the audio sending device and the audio receiving device on the data transmission size, the audio signal must be compressed by the audio encoder in the audio sending device before being transmitted to the audio receiving device. The compressed audio signal must be decoded by the audio decoder in the audio receiving device before it can be played. It can be seen that the popularization of wireless Bluetooth devices has also promoted the vigorous development of various Bluetooth audio codecs.

[0105] Currently, Bluetooth audio codecs include sub-band coding (SBC), advanced audio coding (AAC), aptX series encoders, low-latency hi-definition audio codec (LHDC), low-power and low-latency LC3 audio codec, and LC3plus.

[0106] It should be understood that the audio encoding and decoding method provided in the embodiments of the present application can be applied to an audio sending device (ie, encoding end) and an audio receiving device (ie, decoding end) in a Bluetooth interconnection scenario.

[0107] Figure 1 is a schematic diagram of a Bluetooth interconnection scenario provided by an embodiment of the present application. Referring to Figure 1, the audio sending device in the Bluetooth interconnection scenario can be a mobile phone, a computer, a tablet, etc. Among them, the computer can be a laptop computer, a desktop computer, etc., and the tablet can be a handheld tablet, a car-mounted tablet, etc. The audio receiving device in the Bluetooth interconnection scenario can be a TWS headset, a smart speaker, a wireless headset, a wireless neckband headset, a smart watch, smart glasses, a smart car-mounted device, etc. In other embodiments, the audio receiving device in the Bluetooth interconnection scenario can also be a mobile phone, a computer, a tablet, etc.

[0108] It should be noted that the audio encoding and decoding method provided in the embodiment of the present application can be applied to other device interconnection scenarios in addition to Bluetooth interconnection scenarios. In other words, the system architecture and business scenarios described in the embodiment of the present application are intended to more clearly illustrate the technical solutions of the embodiment of the present application and do not constitute a limitation on the technical solutions provided in the embodiment of the present application. Those skilled in the art will know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiment of the present application are also applicable to similar technical problems.

[0109] FIG2 is a system framework diagram of the audio signal processing method provided in an embodiment of the present application. Referring to FIG2 , the system includes an encoding end and a decoding end. The encoding end includes an input module, an encoding module, and a transmitting module. The decoding end includes a receiving module, an input module, a decoding module, and a playback module.

[0110] On the encoder side, users select one of two encoding modes based on the usage scenario: low-latency encoding mode and high-quality encoding mode. The encoding frame lengths for these two modes are 5ms and 10ms, respectively. For example, if the usage scenario is gaming, live streaming, or making phone calls, users can choose low-latency encoding mode, while if the usage scenario is listening to music through headphones or speakers, users can choose high-quality encoding mode. Users also need to provide the audio signal to be encoded (pulse code modulation (PCM) data, as shown in Figure 2) to the encoder side. Furthermore, users need to set the target bit rate for the resulting encoded stream, i.e., the encoding bit rate for the audio signal. A higher target bit rate indicates better sound quality, but worse interference immunity during short-distance transmission. A lower target bit rate indicates worse sound quality, but better interference immunity during short-distance transmission. Simply put, the encoder's input module receives the encoding frame length, encoding bit rate, and audio signal to be encoded, as submitted by the user.

[0111] The input module of the encoding end inputs the data submitted by the user into the frequency domain encoder of the encoding module.

[0112] The frequency domain encoder in the encoding module encodes the received data to generate a bitstream. The frequency domain encoder analyzes the audio signal to be encoded to determine its characteristics (including mono / dual channel, stationary / non-stationary, full-bandwidth / narrow-bandwidth signal, subjective / objective, etc.). Based on the signal characteristics and the bitrate (i.e., the encoding bitrate), the encoding module enters the corresponding encoding processing submodule, which encodes the audio signal and the packet header of the packet stream (including sampling rate, number of channels, encoding mode, frame length, etc.), ultimately generating the bitstream.

[0113] The sending module of the encoding end sends the code stream to the decoding end. Optionally, the sending module is a short-distance sending module as shown in FIG2 or other types of sending modules, which are not limited in the embodiment of the present application.

[0114] At the decoding end, after receiving the bitstream, the receiving module at the decoding end sends the bitstream to the frequency domain decoder of the decoding module and notifies the input module at the decoding end to obtain the configured bit depth and channel decoding mode. Optionally, the receiving module is a short-range receiving module as shown in Figure 2 or another type of receiving module, which is not limited in this embodiment of the present application.

[0115] The input module of the decoding end inputs the acquired information such as bit depth and channel decoding mode into the frequency domain decoder of the decoding module.

[0116] The frequency domain decoder in the decoding module decodes the bitstream based on the bit depth, channel decoding mode, and other factors to obtain the required audio data (PCM data as shown in Figure 2). The obtained audio data is then sent to the playback module, which plays the audio. The channel decoding mode indicates the channels to be decoded.

[0117] FIG3 is a diagram of an overall framework of an audio codec provided by an embodiment of the present application. Referring to FIG3 , the encoding process at the encoding end includes the following steps:

[0118] (1) PCM input module

[0119] Input PCM data, which can be mono or dual-channel data, with a bit depth of 16 bits, 24 bits, 32 bits floating point, or 32 bits fixed point. Optionally, the PCM input module converts the input PCM data to a uniform bit depth, such as 24 bits, deinterleaves the PCM data, and arranges it according to the left and right channels.

[0120] (2) Adding a low-latency analysis window and improving the discrete cosine transform (MDCT) module

[0121] The PCM data processed in step (1) is subjected to a low-delay analysis window and MDCT transformation to obtain spectrum data in the MDCT domain. The purpose of windowing is to prevent spectrum leakage.

[0122] (3) MDCT domain signal analysis & adaptive bandwidth detection module

[0123] The MDCT domain signal analysis module is effective in the full bit rate scenario, and the adaptive bandwidth detection module is activated in the low bit rate (such as bit rate <150kbps / channel). First, bandwidth detection is performed on the spectrum data of the MDCT domain obtained in the above step (2) to obtain the cutoff frequency or effective bandwidth. Secondly, signal analysis is performed on the spectrum data within the effective bandwidth, that is, whether the frequency point distribution is concentrated or uniform to obtain the energy concentration. Based on the energy concentration, a flag indicating whether the audio signal to be encoded is an objective signal or a subjective signal is obtained (the flag of the objective signal is 1, and the flag of the subjective signal is 0). If it is an objective signal, the scaling factor is not subjected to spectral noise shaping (SNS) processing and MDCT spectrum smoothing at a low bit rate, because this will reduce the coding effect of the objective signal. Then, based on the bandwidth detection result and the subjective and objective signal flags, it is determined whether to perform the sub-band cutoff operation in the MDCT domain. If the audio signal is an objective signal, no subband cutoff operation is performed; if the audio signal is a subjective signal and the bandwidth detection result is marked as 0 (full bandwidth), the subband cutoff operation is determined by the bit rate; if the audio signal is a subjective signal and the bandwidth detection result is non-zero (i.e., a limited bandwidth with a bandwidth less than half the sampling rate), the subband cutoff operation is determined by the bandwidth detection result.

[0124] (4) Subband division selection and scale factor calculation module

[0125] Based on the bit rate level and the subjective and objective signal indicators and cutoff frequency obtained in step (3), the optimal sub-band division method is selected from multiple sub-band division methods, and the total number of sub-bands required to encode the audio signal is obtained. At the same time, the envelope of the spectrum is calculated, that is, the scaling factor corresponding to the selected sub-band division method is calculated.

[0126] (5) MS channel conversion module

[0127] For the dual-channel PCM data, joint coding determination is performed based on the scaling factor calculated in step (4), that is, whether to perform MS channel conversion on the left and right channel data.

[0128] (6) Spectral smoothing module and frequency domain noise shaping module based on scaling factor

[0129] The spectral smoothing module performs MDCT spectral smoothing based on the low bitrate setting (e.g., bitrate <150kbps / channel). The frequency domain noise shaping module performs frequency domain noise shaping on the spectrally smoothed data based on the scaling factor to generate an adjustment factor, which is used to quantize the spectral values ​​of the audio signal. The low bitrate setting is controlled by the low bitrate discrimination module. If the low bitrate setting is not met, spectral smoothing and frequency domain noise shaping are not required.

[0130] (7) Scale factor encoding module

[0131] The scale factors of the plurality of sub-bands are differentially coded or entropy coded according to the distribution of the scale factors.

[0132] (8) Bit allocation & MDCT spectrum quantization and entropy coding module

[0133] Based on the scaling factor obtained in step (4) and the adjustment factor obtained in step (6), the coding is controlled to a constant bit rate (CBR) coding mode through a coarse estimation and a fine estimation bit allocation strategy, and the MDCT spectrum value is quantized and entropy coded.

[0134] (9) Residual coding module

[0135] If the bit consumption in step (8) has not reached the target bit, the uncoded sub-bands are further sorted by importance, and bits are preferentially allocated to the coding of the MDCT spectrum values ​​of the important sub-bands.

[0136] (10) Stream header information packaging module

[0137] The packet header information includes audio sampling rate (such as 44.1kHz / 48kHz / 88.2kHz / 96kHz), channel information (such as mono and dual channels), coding frame length (such as 5ms and 10ms), coding mode (such as time domain, frequency domain, time domain to frequency domain or frequency domain to time domain mode), etc.

[0138] (11) Bit stream (i.e. code stream) sending module

[0139] The code stream includes a header, side information, and a payload. The header carries the header information, which is described in step (10). The side information includes the coding code stream of the scaling factor, the selected subband division method, the cutoff frequency, the low bit rate flag, the joint coding discrimination information (i.e., the MS transform flag), the quantization step size, and other information. The payload includes the coding code stream of the MDCT spectrum and the residual coding code stream.

[0140] The decoding process at the decoding end includes the following steps:

[0141] (1) Stream header information parsing module

[0142] The packet header information is parsed from the received bitstream. The packet header information includes the sampling rate, channel information, encoding frame length, encoding mode and other information of the audio signal. The encoding bitrate is calculated based on the bitstream size, sampling rate and encoding frame length, that is, the bitrate gear information is obtained.

[0143] (2) Scale factor decoding module

[0144] The side information is decoded from the bitstream, including the selected subband division method, cutoff frequency information, low bit rate flag, joint coding discrimination information, quantization step size, and scaling factors of each subband.

[0145] (3) Frequency domain noise shaping module based on scaling factor

[0146] At low bitrates (e.g., encoding rates less than 300kbps, or 150kbps / channel), frequency-domain noise shaping is also required based on the scaling factor to generate an adjustment factor, which is used to dequantize the code values ​​of the spectral values. The low bitrate setting is controlled by the low bitrate discrimination module. If the low bitrate setting is not met, frequency-domain noise shaping is not required.

[0147] (4) MDCT spectrum decoding module and residual decoding module

[0148] The MDCT spectrum decoding module decodes the MDCT spectrum data in the bitstream based on the subband division information, quantization step size information, and scaling factor obtained in step (2). Hole filling is performed at a low bit rate. If the calculated bits are still remaining, the residual decoding module performs residual decoding to obtain the MDCT spectrum data of other subbands, and then the final MDCT spectrum data.

[0149] (5)LR channel conversion module

[0150] According to the side information obtained in step (2), if it is determined according to the joint coding discrimination that the dual-channel joint coding mode is in the dual-channel joint coding mode and the decoding low power consumption mode is not in the decoding low power consumption mode (for example, the coding bit rate is greater than or equal to 300 kbps and the sampling rate is greater than 88.2 kHz), the MDCT spectrum data obtained in step (4) is subjected to LR channel transformation.

[0151] (6) Inverse MDCT transform & low-delay synthesis window module and overlap-add module

[0152] The inverse MDCT transform module performs an inverse MDCT transform on the obtained MDCT spectrum data based on step (4) and step (5) to obtain a time domain aliasing signal, and then adds a low-delay synthesis window to the time domain aliasing signal by the low-delay synthesis window module. The overlap-add module superimposes the time domain aliasing buffer signal of the current frame and the previous frame to obtain a PCM signal, that is, the final PCM data is obtained by overlap-addition.

[0153] (7) PCM output module

[0154] Outputs PCM data of the corresponding channel according to the configured bit depth and channel decoding mode.

[0155] It should be noted that the audio codec framework shown in FIG3 is only an example of a terminal in an embodiment of the present application and is not used to limit the embodiment of the present application. Those skilled in the art can derive other codec frameworks based on FIG3.

[0156] Please refer to Figure 4, which is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Optionally, the electronic device is any device shown in Figure 1, and includes one or more processors 401, a communication bus 402, a memory 403, and one or more communication interfaces 404.

[0157] Processor 401 is a general-purpose central processing unit (CPU), a network processing processor (NP), a microprocessor, or one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. Optionally, the PLD is a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0158] Communication bus 404 is used to transmit information between the above components. Optionally, communication bus 402 is divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or one type of bus.

[0159] Optionally, the memory 403 is a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compact disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 403 exists independently and is connected to the processor 401 via the communication bus 402, or the memory 403 is integrated with the processor 401.

[0160] Communication interface 404 uses any transceiver-like device for communicating with other devices or communication networks. Communication interface 404 includes a wired communication interface and, optionally, a wireless communication interface. Examples of wired communication interfaces include Ethernet interfaces. Optionally, the Ethernet interface is an optical interface, an electrical interface, or a combination thereof. The wireless communication interface is a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0161] Optionally, in some embodiments, the electronic device includes multiple processors, such as processor 401 and processor 405 shown in Figure 4. Each of these processors is a single-core processor or a multi-core processor. Optionally, the processor here refers to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0162] In a specific implementation, as an embodiment, the electronic device further includes an output device 406 and an input device 407. The output device 406 communicates with the processor 401 and can display information in a variety of ways. For example, the output device 406 is a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 407 communicates with the processor 401 and can receive user input in a variety of ways. For example, the input device 407 is a mouse, a keyboard, a touch screen device, or a sensor device.

[0163] In some embodiments, the memory 403 is used to store program code 410 for executing the solution of the present application, and the processor 401 is capable of executing the program code 410 stored in the memory 403. The program code includes one or more software modules, and the electronic device can implement the audio signal processing method provided in the embodiment of FIG. 5 below through the processor 401 and the program code 410 in the memory 403.

[0164] FIG5 is a flowchart of a method for processing an audio signal provided by an embodiment of the present application, which is applied to an encoding end. Referring to FIG5 , the method includes the following steps.

[0165] Step 501: Sub-band division is performed on an audio signal according to multiple sub-band division modes and cutoff sub-bands corresponding to the multiple sub-band division modes to obtain multiple candidate sub-band sets. The multiple candidate sub-band sets correspond one-to-one to the multiple sub-band division modes, and each candidate sub-band set includes multiple sub-bands.

[0166] In an embodiment of the present application, in order to select the best sub-band division method from multiple sub-band division methods, the encoding end first sub-band divides the audio signal according to the multiple sub-band division methods and the cutoff sub-bands corresponding to the multiple sub-band division methods to obtain multiple candidate sub-band sets.

[0167] Taking any of the multiple subband division schemes as an example, the total number of subbands indicated by the subband division scheme is 32, and the corresponding cutoff subband is 16, indicating that the cutoff frequency of the audio signal is in the 16th subband. Taking the full bandwidth of the audio signal as 16 kilohertz (kHz), for example, the cutoff frequency of the audio signal indicated by the cutoff subband is 5 kHz. After the audio signal is subband-divided according to the subband division scheme, the resulting candidate subband set includes a total of 16 subbands. These 16 subbands cover a frequency range of 0-5 kHz, that is, covering the range [0-cutoff frequency].

[0168] It should be noted that the sub-band division process is performed on each audio frame. The audio signal in this article can be considered as an audio frame. Of course, the encoder can perform sub-band division for each audio frame according to this solution.

[0169] In the embodiment of the present application, there are many implementation methods for the encoding end to obtain the cutoff sub-band, and one of the implementation methods is introduced here.

[0170] If the encoding bit rate of the audio signal is less than the first bit rate threshold, the encoder performs bandwidth detection on the spectrum of the audio signal to obtain the cutoff frequency of the audio signal. Based on the cutoff frequency, the encoder determines the cutoff sub-bands corresponding to the multiple sub-band division methods. It should be understood that when the encoding bit rate is low, due to the small number of allocable encoding bits, the encoder determines the cutoff frequency and, therefore, the cutoff sub-band through bandwidth detection. Therefore, the spectrum values ​​exceeding the cutoff frequency are not subsequently encoded, thereby meeting the encoding bit rate requirement while ensuring the encoding effect.

[0171] There are many ways to perform bandwidth detection on the encoder side. In one implementation, since the values ​​of the frequency points after the cutoff frequency in the audio signal spectrum are zero, the encoder side sequentially traverses the values ​​of the frequency points in the spectrum from high to low frequencies. The first frequency point found that is greater than the energy threshold is the cutoff frequency of the audio signal.

[0172] Optionally, the encoder takes the logarithm (e.g., log10) of each frequency point in the spectrum, then iterates through the logarithmized frequency points in sequence from high frequency to low frequency, and determines the first frequency point whose logarithmized value is greater than an energy threshold as the cutoff frequency of the audio signal. Optionally, the energy threshold is -50dB, -80dB, or another value.

[0173] In addition, if the audio signal is a monophonic signal, the encoder performs bandwidth detection on the monophonic spectrum of the audio signal to obtain the cutoff frequency of the audio signal. If the audio signal is a dual-channel signal, the encoder performs bandwidth detection on the left and right channel spectra of the audio signal to obtain the left and right channel cutoff frequencies, respectively. If the left and right channel cutoff frequencies do not match, the encoder determines the larger of the left and right channel cutoff frequencies as the cutoff frequency of the audio signal. If the left and right channel cutoff frequencies match, the encoder determines the left channel cutoff frequency as the cutoff frequency of the audio signal.

[0174] In some other embodiments, the encoding end may also perform bandwidth detection on the spectrum in other ways, which is not limited in this solution.

[0175] Optionally, after obtaining the cutoff frequency of the audio signal, the encoder determines the cutoff sub-bands corresponding to the multiple sub-band division modes respectively based on the position of the cutoff frequency in the complete bandwidth of the audio signal.

[0176] Exemplarily, the cutoff spectrum is located at the 30th frequency point in the full bandwidth of the audio signal, and the 30th frequency point is located in the kth subband among the multiple subbands indicated by a certain subband division method, then the cutoff subband corresponding to the subband division method is k.

[0177] Optionally, in an embodiment of the present application, the first bit rate threshold is 150kbps or other values. The following text will be introduced using the first bit rate threshold of 150kbps as an example. Optionally, in an embodiment of the present application, the encoding bit rate of the audio signal refers to the encoding bit rate of a single channel signal, that is, the encoding bit rate of a single channel is compared with the first bit rate threshold. In some other embodiments, the first bit rate threshold may also be other values. Taking the first bit rate threshold of 150kbps as an example, when the audio signal is a dual-channel signal, the encoding bit rate of the audio signal refers to the encoding bit rate of the left channel or the encoding bit rate of the right channel. Normally, the encoding bit rate of the left channel is the same as the encoding bit rate of the right channel. Then, the encoding end can compare the encoding bit rate of the left channel with 150kbps.

[0178] Of course, in some other embodiments, when the audio signal is a two-channel signal, the encoding bit rate of the audio signal refers to the encoding bit rate of the two channels, and accordingly, the first bit rate threshold is 300 kbps.

[0179] Optionally, if the encoding bit rate of the audio signal is not less than a first bit rate threshold, the encoder determines the last subband indicated by each of the multiple subband division methods as the cutoff subband corresponding to each subband division method. It should be understood that when the encoding bit rate is high, the encoding bit rate requirement can be met even without bandwidth detection due to the large number of allocable encoding bits, thereby improving encoding efficiency to a certain extent. Of course, in other embodiments, the encoder may also perform bandwidth detection on the spectrum of an audio signal whose encoding bit rate is not less than the first bit rate threshold.

[0180] In an embodiment of the present application, before the encoder performs sub-band division on the audio signal according to the multiple sub-band division schemes and the cutoff sub-bands corresponding to the multiple sub-band division schemes, it performs feature analysis on the spectrum of the audio signal to obtain a feature analysis result. Based on the feature analysis result and the encoding bit rate of the audio signal, the encoder determines the multiple sub-band division schemes from the multiple candidate sub-band division schemes. In other words, the encoder preliminarily selects multiple sub-band division schemes from the multiple candidate sub-band division schemes through frequency domain feature analysis, and then subsequently selects the optimal sub-band division scheme from the multiple sub-band division schemes.

[0181] Optionally, the feature analysis result includes a subjective signal flag or an objective signal flag, wherein the subjective signal flag indicates that the energy concentration of the audio signal is not greater than a concentration threshold, and the objective signal flag indicates that the energy concentration of the audio signal is greater than the concentration threshold. That is, the feature analysis includes subjective and objective signal analysis, and the encoder preliminarily selects multiple sub-band division methods based on the subjective and objective signal analysis results and the encoding bit rate.

[0182] Next, an implementation method of subjective and objective signal analysis is introduced.

[0183] In the embodiment of the present application, the encoding end performs subjective and objective signal analysis based on the portion of the spectrum of the audio signal that does not exceed the cutoff frequency, thereby reducing the amount of calculation and improving efficiency while ensuring accuracy.

[0184] The encoding end takes the log10 logarithm of the value of each frequency point that does not exceed the cutoff frequency in the spectrum to obtain the logarithmic result of each frequency point. The encoding end normalizes the logarithmic result of each frequency point to the dBFS scale to obtain the logarithmic result of each frequency point under the dBFS scale. The encoding end determines the first number of frequency points and the second number of frequency points, wherein the first number of frequency points refers to the total number of frequency points whose logarithmic results under the dBFS scale are not greater than the energy threshold, and the second number of frequency points refers to the total number of frequency points in the spectrum that do not exceed the cutoff frequency. The encoding end determines the ratio of the first number of frequency points to the second number of frequency points as the energy concentration of the audio signal. If the energy concentration of the audio signal is greater than the concentration threshold, the encoding end determines that the audio signal is an objective signal and outputs an objective signal flag. If the total energy of the audio signal is not greater than the concentration threshold, the encoding end determines that the audio signal is a subjective signal and outputs a subjective signal flag.

[0185] For example, the encoding end takes the log10 logarithm of the value of each frequency point that does not exceed the cutoff frequency in the spectrum according to formula (1) to obtain the logarithm result of each frequency point. 10 (abs(X(k))),k=0,1,...,cutOffFreq (1)

[0186] In formula (1), X(k) represents the value of the kth frequency point, that is, the kth spectrum value, cutOffFreq represents the frequency point corresponding to the cutoff frequency, that is, the second frequency point number, abs() represents taking the absolute value, and Xlg(k) represents the logarithmic result of the kth frequency point.

[0187] The encoder normalizes the logarithmic results of each frequency point to the dBFS scale according to formula (2) to obtain the logarithmic results of each frequency point in the dBFS scale. 10 (abs(Xmax )), k=0,1,...,cutOffFreq (2)

[0188] In formula (2), XdBFS(k) represents the logarithmic result of the kth frequency point in dBFS scale, X max Indicates the maximum spectrum value in the spectrum that does not exceed the cutoff frequency.

[0189] The encoder counts the total number of frequency points whose logarithmic results on the dBFS scale are no greater than -80 dB to obtain a first frequency point number, lowEnergyCnt. -80 dB represents an energy threshold, which is obtained through statistics or other methods. The encoder determines the energy concentration of the audio signal, energyRate, according to formula (3).

[0190] The encoding end outputs the subjective and objective signal flag objFlag according to formula (4). Wherein, objFlag is 1, indicating the objective signal flag; objFlag is 0, indicating the subjective signal flag.

[0191] In formula (4), threshold represents the concentration threshold.

[0192] In the embodiment of the present application, the concentration threshold is 0.6, which is obtained by statistics or other means, such as being a constant parameter obtained by grading signal distributions of different bandwidths. Of course, in other embodiments, the concentration threshold may also be other values.

[0193] It should be understood that the above example is only one implementation of subjective and objective signal analysis and is not intended to limit the embodiments of the present application.

[0194] In another implementation, after obtaining the first frequency point number and the second frequency point number, the encoding end determines the ratio of the second frequency point number to the first frequency point number as the energy concentration of the audio signal. If the energy concentration of the audio signal is less than the concentration threshold, the encoding end determines that the audio signal is an objective signal and outputs an objective signal flag. If the total energy of the audio signal is not less than the concentration threshold, the encoding end determines that the audio signal is a subjective signal and outputs a subjective signal flag. The concentration threshold in this implementation is the reciprocal of the concentration threshold in the previous implementation. That is, from the opposite side, the frequency domain characteristics of the objective signal are characterized by the fact that the proportion of non-background noise energy (i.e., the first frequency point number) is less than a certain threshold. The essence of this implementation is the same as that of the previous implementation.

[0195] In another implementation, the encoder does not normalize the logarithmic results of each frequency point to the dBFS scale. Instead, it directly determines a third number of frequency points. The third number of frequency points is the total number of frequency points whose logarithmic results are no greater than the energy threshold. The encoder then determines the energy concentration threshold of the audio signal by the ratio of the third number of frequency points to the second number of frequency points. It should be noted that the energy threshold in this implementation differs from the dBFS energy threshold in the first implementation.

[0196] In another implementation, the encoder does not calculate the log10 logarithm of the values ​​of each frequency point in the spectrum that does not exceed the cutoff frequency. Instead, it directly counts the total number of frequency points in the spectrum that do not exceed the energy threshold within the spectral range that does not exceed the cutoff frequency to obtain a fourth frequency point number. The encoder then determines the ratio of the fourth frequency point number to the second frequency point number as the energy concentration threshold for the audio signal. It should be noted that the energy threshold and concentration threshold in this implementation differ from those in the aforementioned implementations.

[0197] It should be understood that taking the log10 logarithm and normalizing to the dBFS scale is for conversion to different scales for operation. The conversion scale is an optional operation on the encoder side. The energy threshold and concentration threshold are different at different scales.

[0198] Next, we will introduce the implementation process of the encoder to preliminarily select multiple sub-band division methods based on the feature analysis results and the encoding bit rate.

[0199] In an embodiment of the present application, the feature analysis result includes a subjective signal flag or an objective signal flag. When the frame length of the audio signal is 10 milliseconds (ms) and the sampling rate is 88.2 kilohertz (kHz) or 96 kHz; or, the frame length of the audio signal is 5 ms and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 ms and the sampling rate is 44.1 kHz or 48 kHz, if the encoding bit rate of the audio signal is less than a first bit rate threshold and the feature analysis result includes the subjective signal flag, the encoder determines the first group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods.The first group of subbands is divided as follows: { {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142, 160,178,196,217,238,259,280,480}, {0,1,2,3,5,7,9,12,15,18,22,26,30,35,41,48,56,65,74,84,94,106,118,134,150, 166,184,202,220,240,260,280,480}, {0,1,2,3,4,5,7,9,11,14,17,21,25,29,34,40,46,52,60,68,76,86,98,110,126,144, 162,180,200,224,250,280,480}, {0,2,4,6,8,12,16,21,26,31,36,41,46,51,56,61,66,71,77,83,89,95,103,111,121, 131,147,163,179,203,240,280,480}, {0,1,2,3,5,7,9,12,15,19,23,27,32,37,43,49,57,66,76,86,98,110,125,140,158, 176,194,216,238,264,290,320,480}, {0,1,2,3,5,7,10,13,17,21,25,30,35,41,47,54,62,70,80,90,102,114,130,146,162, 180,198,218,240,264,290,320,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,66,76,88,100,112,128,144,160,182, 204,226,256,286,316,352,400,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,68,78,90,102,116,132,148,166,186, 208,234,262,292,324,360,400,480}}.

[0200] FIG6 shows the relationship between the sub-bands indicated by the eight sub-band division modes included in the first group of sub-band division modes and the initial frequency points of the sub-bands.

[0201] When the frame length of the audio signal is 10 ms and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 ms and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 ms and the sampling rate is 44.1 kHz or 48 kHz, if the encoding bit rate of the audio signal is not less than the first bit rate threshold, and / or the feature analysis result includes an objective signal flag, the encoder determines the second group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods.The second group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,26,30,35,40,45,50,57,64,73,82,92,102, 112,124,136,148,160,480}, {0,1,2,3,4,5,7,9,11,13,15,18,21,24,28,33,38,44,50,57,64,73,82,93,104,116, 128,140,155,170,185,200,480}, {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142, 160,178,196,217,238,259,280,480}, {0,1,2,4,6,10,14,18,22,26,30,34,42,50,58,66,74,84,96,108,120,136,152,168, 192,216,240,272,304,336,376,424,480}, {0,1,2,4,6,10,14,18,26,34,42,50,62,74,86,98,112,128,144,160,176,196,216,236, 256,280,304,328,352,384,416,448,480}, {0,80,92,104,112,120,128,136,144,148,152,156,160,164,168,172,176,180,184, 188,192,196,200,208,216,224,232,240,248,256,268,280,480}, {0,200,212,224,232,240,248,256,264,268,272,276,280,284,288,292,296,300,304, 308,312,316,320,328,336,344,352,360,368,376,388,400,480}, {0,320,332,344,356,364,372,380,384,388,392,396,400,404,408,412,416,420,424, 428,432,436,440,444,448,452,456,460,464,468,472,476,480}}.

[0202] FIG7 shows the relationship between the sub-bands indicated by the eight sub-band division methods included in the second group of sub-band division methods and the initial frequency points of the sub-bands.

[0203] It should be noted that when the audio signal frame length is 10 ms and the sampling rate is 88.2 kHz or 96 kHz, the spectrum of each audio frame included in the audio signal includes 960 frequency points. During subband division according to the second group of subband division methods, the encoder multiplies each subband division value in the second group of subband division methods by 2 to obtain subband division values ​​corresponding to 960 frequency points, and then performs subband division according to the subband division values ​​corresponding to 960 frequency points. When the audio signal frame length is 5 ms and the sampling rate is 88.2 kHz or 96 kHz, or when the audio signal frame length is 10 ms and the sampling rate is 44.1 kHz or 48 kHz, since the spectrum of each audio frame included in the audio signal includes 480 frequency points, the last subband division value in each subband division method included in the second group of subband division methods is also 480. Therefore, the encoder can directly perform subband division according to the second group of subband division methods.

[0204] When the frame length of the audio signal is 5 ms and the sampling rate is 44.1 kHz or 48 kHz, if the encoding bit rate of the audio signal is less than the first bit rate threshold and the feature analysis result includes a subjective signal flag, the encoder determines the third group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods. The third group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,12,14,16,19,22,26,30,35,39,44,50,56,63,71,80,89, 98,108,119,129,140,240}, {0,1,2,3,4,5,6,7,8,9,11,13,15,17,20,24,28,32,37,42,47,53,59,67,75,83,92, 101,110,120,130,140,240}, {0,1,2,3,4,5,6,7,8,9,10,11,12,14,17,20,23,26,30,34,38,43,49,55,63,72,81,90, 100,112,125,140,240}, {0,1,2,3,4,6,8,10,13,15,18,20,23,25,28,30,33,35,38,41,44,47,51,55,60,65,73, 81,89,101,120,140,240}, {0,1,2,3,4,5,6,7,8,10,12,14,17,20,23,27,31,35,40,45,51,57,65,73,81,90,99, 109,120,132,145,160,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,33,38,44,50,56,64,72,80,91,102,113, 128,143,158,176,200,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,34,39,45,51,58,66,74,83,93,104,117, 131,146,162,180,200,240}}.

[0205] FIG8 shows the relationship between the sub-bands indicated by the eight sub-band division methods included in the third group of sub-band division methods and the initial frequency points of the sub-bands.

[0206] When the frame length of the audio signal is 5 ms and the sampling rate is 44.1 kHz or 48 kHz, if the encoding bit rate of the audio signal is not less than the first bit rate threshold, and / or the feature analysis result includes the objective signal flag, the encoder determines the fourth group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods. The fourth group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,30, 32,34,37,40,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20,22,24,26,28,30,32,34,36,38, 41,44,47,50,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,17,19,21,24,27,30,34,38,42,48,54,60,68, 76,84,94,106,120}, {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,25,28,32,36,40,44,49,54,59,64,70,76,82, 88,96,104,112,120}, {0,20,23,26,28,30,32,34,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,52,54, 56,58,60,62,64,67,70,120}, ,98,99,100,101,102,103,104,105,106,107,108,109, 110,111,112,113,114,115,116,117,118,119,120}}.

[0207] FIG9 shows the relationship between the sub-bands indicated by the eight sub-band division methods included in the fourth group of sub-band division methods and the initial frequency points of the sub-bands.

[0208] It should be noted that when the frame length of the audio signal is 5ms and the sampling rate is 44.1kHz or 48kHz, the spectrum of each audio frame included in the audio signal includes 240 frequency points. In the process of sub-band division according to the fourth group of sub-band division methods, the encoding end multiplies each sub-band division value in the above-mentioned fourth group of sub-band division methods by 2 to obtain a sub-band division value corresponding to 240 frequency points, and performs sub-band division according to the sub-band division value corresponding to the 240 frequency points.

[0209] It should be noted that each sub-band division method provided in the embodiments of the present application complies with the Bark requirement. The Bark scale refers to a spectrum sub-band division strategy that divides the sub-bands based on the auditory perception characteristics of the human ear.

[0210] Step 502: Determine a total scale value of each candidate subband set based on the spectral value of the audio signal in the subbands included in each candidate subband set, the encoding bit rate of the audio signal, and the subband bandwidth of the subbands included in each candidate subband set.

[0211] In an embodiment of the present application, after obtaining multiple candidate subband sets corresponding one-to-one to the multiple subband division methods, the encoder determines the total scale value of each candidate subband set based on the spectral value of the audio signal in the subband included in each candidate subband set, the encoding bit rate of the audio signal, and the subband bandwidth of the subband included in each candidate subband set.

[0212] Optionally, the encoder determines the total scale value for each candidate subband set based on spectral values ​​of the audio signal within the subbands included in each candidate subband set, the encoding bit rate of the audio signal, and the subband bandwidth of the subbands included in each candidate subband set. The encoder determines, for a first candidate subband set among the multiple candidate subband sets, a scale factor for each subband included in the first candidate subband set based on spectral values ​​of the audio signal within the subbands included in the first candidate subband set. The first candidate subband set is any candidate subband set among the multiple candidate subband sets. The encoder then determines the total scale value for the first candidate subband set based on the encoding bit rate of the audio signal, and the scale factors and subband bandwidths of the subbands included in the first candidate subband set. It should be noted that for each candidate subband set other than the first candidate subband set among the multiple candidate subband sets, the encoder determines the total scale value for each of the other candidate subband sets in the same manner as for determining the total scale value for the first candidate subband set.

[0213] There are many implementation methods for the encoder to determine the scale factor for each subband. In one implementation method, the encoder determines the scale factor for each subband included in the first candidate subband set based on the spectral values ​​of the audio signal in each subband included in the first candidate subband set. The implementation process includes: for a first subband included in the first candidate subband set, the encoder obtains the maximum absolute value of all spectral values ​​of the audio signal in the first subband, and determines the scale factor for the first subband based on the maximum value, where the first subband is any subband in the first candidate subband set. It should be noted that for each subband in the first candidate subband set other than the first subband, the encoder determines the scale factor for each other subband in the first candidate subband set in the same manner as the scale factor for the first subband.

[0214] Exemplarily, the encoder determines the scaling factor of each subband in the first candidate subband set according to formula (5).

[0215] In formula (5), X(k) represents the kth spectrum value of the audio signal, b represents the number of the subband division, I(b) represents the initial frequency point of subband b, B represents the cutoff subband corresponding to the first candidate subband set, that is, the total number of subbands included in the first candidate subband set, abs() represents taking the absolute value, max() represents taking the maximum value, ceil() represents rounding up, and E() represents the subband scale factor.

[0216] Next, an implementation method for determining the total scaling value of the first candidate subband set by the encoder based on the encoding bit rate of the audio signal and the scaling factors and subband bandwidths of the subbands included in the first candidate subband set is introduced.

[0217] Optionally, the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold. The encoder determines an energy smoothing reference value based on the encoding bit rate and a second bit rate threshold of the audio signal. The encoder determines the total energy value of each subband included in the first candidate subband set based on the energy smoothing reference value, the scale factor, and the subband bandwidth of each subband included in the first candidate subband set. The encoder adds the total energy values ​​of each subband included in the first candidate subband set to obtain a total scale value for the first candidate subband set. The fact that the energy concentration of the audio signal is greater than the concentration threshold indicates that the audio signal is an objective signal. It should be understood that when the encoding bit rate is relatively high and / or the audio signal is an objective signal, the encoder determines the total scale value based on the total energy value of each subband.

[0218] There are many ways to implement the energy smoothing reference value at the encoder end. Here, one of the implementations is introduced. In this implementation, the encoder end determines the energy smoothing reference value according to formula (6). floor =int[min(((200-bpsPerChn)),0)] (6)

[0219] In formula (6), E floor represents the energy smoothing baseline value, bpsPerChn represents the audio signal's encoding bitrate, where the audio signal's encoding bitrate refers to the encoding bitrate of a single channel. 200 indicates that the second bitrate threshold is 200 kbps. min() indicates the minimum value, and int() indicates rounding down. It should be noted that the second bitrate threshold can also be other values.

[0220] There are multiple implementations for determining the total energy value of each subband in the first candidate subband set based on the energy smoothing reference value, the scale factor, and the subband bandwidth of each subband in the first candidate subband set. One implementation is described here. In this implementation, for the first subband in the first candidate subband set, the encoder determines the maximum value of the first subband's scale factor and the energy smoothing reference value as the reference scale value for the first subband. The encoder then multiplies the reference scale value for the first subband by the subband bandwidth of the first subband as the total energy value for the first subband. The first subband is any subband in the first candidate subband set. It should be noted that for each subband in the first candidate subband set other than the first subband, the encoder determines the total energy value of each subband in the same manner as it determines the total energy value of the first subband.

[0221] The encoder determines the total energy value of each subband included in the first candidate subband set and the total scale value of the first candidate subband set according to formula (7).

[0222] In formula (7), b represents the number of the subband division, B represents the cutoff subband corresponding to the first candidate subband set, bandWidth() represents the subband bandwidth, E(b) represents the scaling factor of subband b, and E floor Indicates the energy smoothing reference value, max() indicates the maximum value, max[E(b),E floor ]*bandWidth(b) represents the total energy value of subband b, E total Indicates the total scale value of the first subband set.

[0223] The above describes the implementation process of determining the total scale value of the first candidate subband set by the encoder when the audio encoding bit rate is not less than the first bit rate threshold and / or the energy concentration of the audio signal is greater than the concentration threshold. Next, the implementation process of determining the total scale value of the first candidate subband set by the encoder when the audio encoding bit rate is less than the first bit rate threshold and the energy concentration of the audio signal is not greater than the concentration threshold will be described.

[0224] If the encoding bit rate of the audio signal is less than a first bit rate threshold and the energy concentration of the audio signal is not greater than the concentration threshold, the encoder determines the total scale value of the first candidate subband set based on the encoding bit rate of the audio signal and the scale factors and subband bandwidths of the subbands included in the first candidate subband set. The process includes: the encoder determines an energy smoothing reference value based on the encoding bit rate of the audio signal and the second bit rate threshold. The encoder determines a scale difference value for each subband included in the first candidate subband set based on the energy smoothing reference value and the scale factors of the subbands included in the first candidate subband set, wherein the scale difference value represents the difference between the scale factor of the corresponding subband and the scale factor of the adjacent subband. The encoder determines the total scale value of the first candidate subband set based on the scale difference values ​​of the subbands included in the first candidate subband set and the subband bandwidth. If the energy concentration of the audio signal is not greater than the concentration threshold, it indicates that the audio signal is a subjective signal. It should be understood that, when the encoding bit rate is low and the audio signal is a subjective signal, the encoding end determines the total scale value based on the difference between each sub-band and the adjacent sub-band.

[0225] The implementation of determining the energy smoothing reference value by the encoder based on the encoding bit rate of the audio signal and the second bit rate threshold is described above and will not be repeated here.

[0226] There are various implementations for determining the scaled difference values ​​of each subband included in the first candidate subband set by the encoder based on the energy smoothing reference value and the scale factors of each subband included in the first candidate subband set. One implementation is described here. In this implementation, for a first subband included in the first candidate subband set, the encoder determines a first smoothed value, a second smoothed value, and a third smoothed value of the first subband based on the energy smoothing reference value, the scale factor of the first subband, and the scale factors of adjacent subbands of the first subband. The encoder determines the scaled difference value of the first subband based on the first smoothed value, the second smoothed value, and the third smoothed value of the first subband. The first subband is any subband in the first candidate subband set.

[0227] Optionally, if the first subband is the first subband in the first candidate subband set, the encoder determines the maximum value of the scaling factor of the first subband and the energy smoothing reference value as the first smoothing value of the first subband; if the first subband is not the first subband in the first candidate subband set, the encoder determines the maximum value of the scaling factor of the previous adjacent subband of the first subband and the energy smoothing reference value as the first smoothing value of the first subband.

[0228] The encoder determines the maximum value of the scaling factor of the first sub-band and the energy smoothing reference value as the second smoothing value of the first sub-band.

[0229] If the first subband is the last subband in the first candidate subband set, the encoder determines the maximum value of the scaling factor of the first subband and the energy smoothing reference value as the third smoothing value of the first subband; if the first subband is not the last subband in the first candidate subband set, the encoder determines the maximum value of the scaling factor of the next adjacent subband after the first subband and the energy smoothing reference value as the third smoothing value of the first subband.

[0230] That is, the encoding end determines the first smoothing value, the second smoothing value, and the third smoothing value of each subband according to formula (8), formula (9), and formula (10) respectively. center(b)=max[E(b),E floor ],for b=0,1,2,...,B-1 (9)

[0231] In formula (8), formula (9), and formula (10), left(), center(), and right() represent the first smoothed value, the second smoothed value, and the third smoothed value, respectively. In the embodiment of the present application, the first smoothed value, the second smoothed value, and the third smoothed value may also be referred to as the left smoothed value, the center smoothed value, and the right smoothed value, respectively.

[0232] Optionally, after the encoder determines the first smoothed value, the second smoothed value, and the third smoothed value of the first subband, the encoder determines a scaled difference value for the first subband, for the first subband included in the first candidate subband set. The encoder determines a first difference value and a second difference value for the first subband, where the first difference value refers to the absolute value of the difference between the first smoothed value and the second smoothed value of the first subband, and the second difference value refers to the absolute value of the difference between the second smoothed value and the third smoothed value of the first subband. The encoder determines the scaled difference value for the first subband based on the first difference value and the second difference value of the first subband. The first subband is any subband in the first candidate subband set.

[0233] Exemplarily, the encoder determines the scale difference value of the first subband according to formula (11). diff=max{8-abs[center(b)-left(b)]-abs[center(b)-right(b)],1}b=0,1,...,B-1 (11)

[0234] After determining the scaled difference value of each subband included in the first candidate subband set, the encoder determines the total scaled value of the first candidate subband set based on the scaled difference value of each subband and the subband bandwidth. The encoder then determines a smoothing weight coefficient for each subband included in the first candidate subband set based on the number of subbands included in the first candidate subband set and the subband bandwidth of each subband. The encoder then adds the smoothing weight coefficients of each subband included in the first candidate subband set to obtain a total smoothing weight coefficient for the first candidate subband set. The encoder then multiplies the scaled difference value of each subband included in the first candidate subband set by the smoothing weight coefficient to obtain a weighted scaled difference value for each subband included in the first candidate subband set. The encoder then adds the weighted scaled difference values ​​of each subband included in the first candidate subband set to obtain a summed scaled value for the first candidate subband set. The encoder then divides the summed scaled value of the first candidate subband set by the total smoothing weight coefficient to obtain a total scaled value for the first candidate subband set.

[0235] Among them, the step of determining the smoothing weight coefficient of each subband and the total smoothing weight coefficient of the first candidate subband set by the encoding end can also be performed before determining the scale difference value of each subband. The embodiment of the present application does not limit the order in which the encoding end performs the various steps.

[0236] Optionally, the encoder determines a scale factor for subband division based on the number of subbands included in the first candidate subband set. The encoder determines a smoothing weighting coefficient for each subband included in the first candidate subband set based on the scale factor for subband division and the subband bandwidth of each subband included in the first candidate subband set.

[0237] Exemplarily, the encoder determines the scale factor coef of the sub-band division according to formula (12).

[0238] The encoder determines the smoothing weight coefficient frac of each band according to formula (13). frac(b)=max{min[bandsWidth(b)*max[(1-b*cofe),0.05],4.0],1.0}b=0,1,...,B-1 (13)

[0239] The encoder determines the total smoothing weight coefficient sum of the first candidate subband set according to formula (14).

[0240] The encoder determines the summed scaling value E′ of the first candidate subband set according to formula (15): total .

[0241] Among them, E diff (b)*frac(b) represents the weighted scaled difference value of subband b.

[0242] The encoder determines the total scale value E of the first candidate subband set according to formula (16): total .

[0243] It should be noted that, when the audio signal is a monophonic signal, the encoder can calculate the total scale value of each candidate subband set according to the above formula. When the audio signal is a dual-channel signal, the spectrum of the audio signal includes a left channel spectrum and a right channel spectrum, and the encoder calculates the total scale value of each candidate subband set based on the left channel spectrum and the right channel spectrum. For example, the encoder adds the total scale value calculated based on the left channel spectrum and the total scale value calculated based on the right channel spectrum to obtain the total scale value of the candidate subband set. In one implementation, a layer of Σ summation is added to the above formula related to Σ summation, and the added layer of Σ summation represents the addition of the relevant data of the left channel and the relevant data of the right channel.

[0244] Step 503: Select a candidate subband set from the multiple candidate subband sets as a target subband set according to the total scale value of each candidate subband set. Each subband included in the target subband set has a scale factor, and the scale factor is used to shape the spectrum envelope of the audio signal.

[0245] In this embodiment of the present application, the encoder determines the candidate subband set with the smallest total scale value among the multiple candidate subband sets as the target subband set. In other embodiments, the encoder may also determine the candidate subband set with the second smallest total scale value among the multiple candidate subband sets as the target subband set, where the second smallest total scale value refers to the smallest total scale value among all scale values ​​excluding the smallest total scale value.

[0246] As can be seen from the above introduction, the encoder selects the best sub-band division method from multiple sub-band division methods according to the characteristics of the audio signal. That is, the sub-band division method in this scheme has the characteristics of signal adaptation, which is conducive to improving the encoding effect and compression efficiency.

[0247] To further improve coding performance and compression efficiency, when the audio signal is a two-channel signal, the encoder can also determine, based on the determined target subband set, whether applying mid / side stereo transform coding (MS transform) to the audio signal's spectrum will help improve coding performance. If the MS transform is determined to improve coding performance, the encoder performs subsequent coding based on the MS-transformed spectrum. If the MS transform is determined not to improve coding performance, the encoder performs subsequent coding based on the original audio signal's spectrum. This will be described below.

[0248] In an embodiment of the present application, when the audio signal is a dual-channel signal, the encoding end determines a first total scale value based on the scale factors and subband bandwidths of each subband included in the target subband set. The encoding end performs an MS transform on the spectrum of the dual-channel signal to obtain the spectrum of the transformed dual-channel signal. The encoding end determines the scale factors of each subband in the target subband set after the transform based on the spectrum values ​​of the dual-channel signal in each subband included in the target subband set. The encoding end determines a second total scale value based on the scale factors and subband bandwidths of each subband included in the target subband set after the transform. If the first total scale value is not greater than the second total scale value, the encoding end determines the dual-channel signal (i.e., the dual-channel signal before the MS transform) as the signal to be encoded.

[0249] It should be understood that the first total scale value is the total scale value before the MS transform, and the second total scale value is the total scale value after the MS transform. A higher total scale value indicates a lower coding performance benefit. If the first total scale value is not greater than the second total scale value, it indicates that the MS transform does not help improve coding performance. Therefore, the encoder determines the pre-MS transform dual-channel signal as the signal to be encoded.

[0250] Optionally, the spectrum of the two-channel signal before the MS transformation is called the LR spectrum, and the spectrum of the two-channel signal after the MS transformation is called the MS spectrum, where LR represents the left and right channels.

[0251] When the audio signal is a two-channel signal, the scale factor includes a left channel scale factor and a right channel scale factor. Optionally, the encoder determines the first total scale value based on the scale factor and subband bandwidth of each subband included in the target subband set. The implementation process includes: the encoder multiplies the left channel scale factor of each subband included in the target subband set by the subband bandwidth of the corresponding subband to determine the left channel energy value of the corresponding subband, and multiplies the right channel scale factor of each subband included in the target subband set by the subband bandwidth of the corresponding subband to determine the right channel energy value of the corresponding subband. The encoder adds the left channel energy values ​​and the right channel energy values ​​of all subbands included in the target subband set to obtain the first total scale value.

[0252] Exemplarily, the encoding end determines the first total scale value according to formula (17).

[0253] In formula (17), totalScale1 represents the first total scale value, ch represents the number of the left and right channels, when ch=0, E(b) represents the scale factor of the left channel, and when ch=1, E(b) represents the scale factor of the right channel.

[0254] The encoding end performs MS transform according to formula (18).

[0255] In formula (18), L and R represent the spectrum value of the left channel and the spectrum value of the right channel before the transformation, respectively. M and S represent the spectrum value of the left channel and the spectrum value of the right channel after the transformation, respectively. It should be noted that the encoding end processes the spectrum values ​​of the corresponding frequency points in the left channel spectrum and the right channel spectrum according to formula (18), thereby obtaining the spectrum values ​​of the corresponding frequency points in the left channel spectrum and the right channel spectrum after the transformation. The spectrum values ​​of the left channel and the spectrum values ​​of the right channel after the transformation mentioned here refer to the spectrum values ​​of the two channels included in the transformed dual-channel signal. The transformed left channel and right channel can also be called the transformed M channel and S channel.

[0256] The encoding end determines the transformed scaling factor of each subband according to formula (19) which is similar to formula (5).

[0257] In formula (19), X_MS(k) represents the kth spectral value after transformation, and E_MS(b) represents the scale factor of subband b in the M channel or S channel, that is, the scale factor of a channel after transformation of subband b. It should be noted that the encoder calculates the scale factor of the M channel based on the spectral value of the M channel according to formula (19), and calculates the scale factor of the S channel based on the spectral value of the S channel according to formula (19).

[0258] The encoding end determines the second total scale value according to formula (20).

[0259] In formula (20), totalScale2 represents the second total scale value, and ch represents the number of the M channel and the S channel. When ch = 0, E_MS(b) represents the scale factor of the transformed left channel of the subband, and when ch = 1, E_MS(b) represents the scale factor of the transformed right channel of the subband, that is, the scale factor of subband b in the M channel or the S channel.

[0260] Optionally, if the first total scale value is greater than the second total scale value, the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold, the encoder determines the transformed two-channel signal as the signal to be encoded. It should be understood that the first total scale value being greater than the second total scale value indicates that the MS transform helps improve encoding performance. Therefore, the encoder determines the MS transformed two-channel signal as the signal to be encoded.

[0261] As can be seen from the foregoing, when the audio signal is a two-channel signal, the scale factors include a left channel scale factor and a right channel scale factor. Optionally, if the first total scale value is greater than the second total scale value, the encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold, the encoder determines the left and right scale factor difference values ​​for each subband included in the target subband set based on the left and right channel scale factors of each subband included in the target subband set. The encoder determines the start and end frequency difference values ​​for each subband included in the target subband set based on the initial frequency point and cutoff frequency point of each subband included in the target subband set. If there is at least one subband in the target subband set whose left and right scale factor difference value is greater than the difference threshold and the start and end frequency difference value is within a first range, the encoder determines the untransformed two-channel signal as the signal to be encoded.

[0262] That is, when the encoding bit rate is low and the audio signal is an objective signal, the encoder determines whether the MS transform improves the encoding performance based on the difference in the left and right channel scaling factors and the difference in the start and end frequencies of the subbands.

[0263] Optionally, the encoding end traverses all subbands in the target self-band set. When it is found through the traversal that the difference value of the left and right scaling factors of a subband is greater than the difference threshold and the start and end frequency difference values ​​are within the first range, the encoding end determines that the dual-channel signal before the transformation is determined as the signal to be encoded.

[0264] For example, the encoder determines the difference between the left and right scaling factors of each subband according to formula (21). diffSFflag(b) = abs[E_L(b) - E_R(b)] (21)

[0265] In formula (21), E_L() represents the left channel scale factor, E_R() represents the left channel scale factor, and diffSFflag() represents the left scale factor difference value.

[0266] When the encoder determines the difference value of the left and right scale factors of each sub-band according to formula (21), the difference threshold is 3.

[0267] The encoding end determines the subband center frequency of each subband according to formula (22).

[0268] In formula (22), freq() represents the difference between the start and end frequencies, bandstart() and bandend() represent the initial frequency point and the cutoff frequency point, respectively. SamplingRate represents the sampling rate in Hz, and FrameLength represents the number of sampling points per frame.

[0269] Optionally, when the encoding end determines the subband center frequency of each subband according to formula (22), the first range is (3500, 12000).

[0270] In simple terms, when the encoder uses formula (21) and formula (22), the encoder traverses all subbands in the target self-band set. When it is found through the traversal that the left and right scale factor difference value diffSFflag of a subband is greater than 3 and the subband center frequency freq is in the interval (3500, 12000], the encoder determines that the dual-channel signal before transformation is determined as the signal to be encoded.

[0271] If the at least one subband does not exist in the target subband set, the encoder determines the transformed binaural signal as the signal to be encoded. The at least one subband is a subband whose left and right scaling factor difference is greater than a difference threshold and whose subband center frequency is within the first range.

[0272] Next, please refer to FIG. 10 to further explain the implementation process of the encoder determining whether to use the transformed two-channel signal as the signal to be encoded.

[0273] Referring to Figure 10, the encoder calculates a first total scale value based on the selected target subband set and the left and right (LR) channel scale factors (SF) of each subband in the target subband set. The first total scale value refers to the sum of the products of the LR channel scale factors of all subbands in the target subband set and the corresponding subband bandwidths. The encoder converts the spectrum of the LR channels into the spectrum of the MS channels and calculates a second total scale value. The second total scale value refers to the sum of the products of the MS channel scale factors of all subbands in the target subband set and the corresponding subband bandwidths. If the first total scale value is not greater than the second total scale value, the encoder determines the dual-channel signal before the transformation as the signal to be encoded and sets MSFlag = 0, indicating that the execution of subsequent operations will not be based on the spectrum value after the MS transformation.

[0274] If the first total scale value is greater than the second total scale value, the encoder determines whether the audio signal (i.e., the dual-channel signal before conversion) satisfies a first condition, where the first condition is that the encoding bit rate of the audio signal is less than a first bit rate threshold and the energy concentration threshold of the audio signal is less than a concentration threshold. If the audio signal satisfies the first condition, the encoder sets the high bit rate flag to 0; if the audio signal does not satisfy the first condition, the encoder sets the high bit rate flag to 1.

[0275] If the high bit rate Flag = 1, the encoder will determine the transformed dual-channel signal as the signal to be encoded, and set MSFlag = 1, indicating that the execution of subsequent operations will be based on the spectrum value after MS transformation. If the high bit rate Flag = 0, the encoder calculates the LR channel SF difference value and the sub-band center frequency of each sub-band by traversal. If a sub-band traversed meets the second condition, the encoder sets the SF difference Flag = 1. The second condition means that the LR channel SF difference value of the corresponding sub-band is less than the difference threshold and the sub-band center frequency is within the first range. If all traversed sub-bands do not meet the second condition, the encoder sets the SF difference Flag = 0.

[0276] If SF Difference Flag = 1, the encoder determines the untransformed binaural signal as the signal to be coded and sets MSFlag = 0. If SF Difference Flag = 0, the encoder determines the transformed binaural signal as the signal to be coded and sets MSFlag = 1.

[0277] It should be noted that, in addition to the above-described implementation method of the encoder determining whether to use the transformed two-channel signal as the signal to be encoded, the encoder may also make the determination in other ways. In other words, the above implementation method is not intended to limit the embodiments of the present application.

[0278] In summary, in an embodiment of the present application, the optimal sub-band division method is selected from a plurality of sub-band division methods according to the characteristics of the audio signal. That is, the sub-band division method has the characteristic of signal adaptation and can adapt to the encoding bit rate of the audio signal, thereby improving the anti-interference ability. Specifically, the audio signal is first divided according to a plurality of sub-band division methods, and then based on the spectral value of the audio signal in each divided sub-band, the bandwidth of each sub-band, and the encoding bit rate of the audio signal, the total scale value corresponding to each sub-band division method is determined. Based on the total scale value, the optimal target sub-band division method is selected, that is, the optimal sub-band set is obtained. If the spectrum envelope is subsequently shaped according to the scaling factor of each sub-band in the optimal sub-band set, the encoding effect and compression efficiency can be improved.

[0279] Figure 11 is a schematic diagram of the structure of an audio signal processing apparatus 1100 provided in an embodiment of the present application. This processing apparatus 1100 can be implemented as part or all of an electronic device using software, hardware, or a combination of both. The electronic device can be any of the devices shown in Figure 1 . Referring to Figure 11 , the apparatus includes a subband division module 1101, a first determination module 1102, and a selection module 1103.

[0280] a subband division module 1101 configured to perform subband division on the audio signal according to a plurality of subband division modes and cutoff subbands corresponding to the plurality of subband division modes, to obtain a plurality of candidate subband sets, wherein the plurality of candidate subband sets correspond one-to-one to the plurality of subband division modes, and each candidate subband set includes a plurality of subbands;

[0281] a first determining module 1102, configured to determine a total scale value for each candidate subband set based on a spectrum value of the audio signal in the subbands included in each candidate subband set, a coding bit rate of the audio signal, and a subband bandwidth of the subbands included in each candidate subband set;

[0282] The selection module 1103 is configured to select a candidate subband set from the plurality of candidate subband sets as a target subband set according to the total scale value of each candidate subband set. Each subband included in the target subband set has a scale factor, and the scale factor is used to shape the spectrum envelope of the audio signal.

[0283] Optionally, the selection module 1103 is used to:

[0284] A candidate subband set with the smallest total scale value among the multiple candidate subband sets is determined as the target subband set.

[0285] Optionally, the first determining module 1102 includes:

[0286] a first determining submodule configured to determine, for a first candidate subband set among the multiple candidate subband sets, a scale factor for each subband included in the first candidate subband set based on spectral values ​​of the audio signal within each subband included in the first candidate subband set, where the first candidate subband set is any candidate subband set among the multiple candidate subband sets;

[0287] The second determining submodule is configured to determine a total scaling value of the first candidate subband set based on a coding rate of the audio signal and a scaling factor and a subband bandwidth of each subband included in the first candidate subband set.

[0288] Optionally, the second determining submodule is configured to:

[0289] For a first subband included in the first candidate subband set, obtaining a maximum absolute value of all spectral values ​​of the audio signal in the first subband, where the first subband is any subband in the first candidate subband set;

[0290] Based on the maximum value, a scale factor for the first subband is determined.

[0291] Optionally, the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold;

[0292] The second determining submodule is used for:

[0293] Determining an energy smoothing reference value based on the encoding rate of the audio signal and a second rate threshold;

[0294] Determining a total energy value of each subband included in the first candidate subband set based on the energy smoothing reference value, the scale factor and the subband bandwidth of each subband included in the first candidate subband set;

[0295] The total energy values ​​of the subbands included in the first candidate subband set are added together to obtain a total scale value of the first candidate subband set.

[0296] Optionally, the second determining submodule is configured to:

[0297] For a first subband included in the first candidate subband set, determining a maximum value between a scale factor of the first subband and an energy smoothing reference value as a reference scale value of the first subband, where the first subband is any subband in the first candidate subband set;

[0298] The product of the reference scale value of the first sub-band and the sub-band bandwidth of the first sub-band is determined as the total energy value of the first sub-band.

[0299] Optionally, the encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold;

[0300] The second determining submodule is used for:

[0301] Determining an energy smoothing reference value based on the encoding rate of the audio signal and a second rate threshold;

[0302] Determining, based on the energy smoothing reference value and the scale factor of each subband included in the first candidate subband set, a scale difference value for each subband included in the first candidate subband set, wherein the scale difference value represents a difference between the scale factor of the corresponding subband and the scale factor of an adjacent subband of the corresponding subband;

[0303] An overall scaling value of the first candidate subband set is determined based on the scaling difference values ​​and subband bandwidths of the subbands included in the first candidate subband set.

[0304] Optionally, the second determining submodule is configured to:

[0305] For a first subband included in the first candidate subband set, determining a first smoothed value, a second smoothed value, and a third smoothed value of the first subband based on an energy smoothing reference value, a scale factor of the first subband, and scale factors of adjacent subbands of the first subband, where the first subband is any subband in the first candidate subband set;

[0306] A scaled difference value for the first subband is determined based on the first smoothed value, the second smoothed value, and the third smoothed value of the first subband.

[0307] Optionally, the second determining submodule is configured to:

[0308] If the first subband is the first subband in the first candidate subband set, determining the maximum value of the scale factor of the first subband and the energy smoothing reference value as the first smoothing value of the first subband; if the first subband is not the first subband in the first candidate subband set, determining the maximum value of the scale factor of the previous adjacent subband of the first subband and the energy smoothing reference value as the first smoothing value of the first subband;

[0309] Determine the maximum value of the scale factor of the first sub-band and the energy smoothing reference value as the second smoothing value of the first sub-band;

[0310] If the first subband is the last subband in the first candidate subband set, the maximum value of the scale factor of the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband; if the first subband is not the last subband in the first candidate subband set, the maximum value of the scale factor of the next adjacent subband after the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband.

[0311] Optionally, the second determining submodule is configured to:

[0312] For a first subband included in the first candidate subband set, determining a first difference value and a second difference value of the first subband, where the first difference value refers to an absolute value of a difference between a first smoothed value and a second smoothed value of the first subband, and the second difference value refers to an absolute value of a difference between a second smoothed value and a third smoothed value of the first subband, where the first subband is any subband in the first candidate subband set;

[0313] A scaled disparity value for the first subband is determined based on the first disparity value and the second disparity value for the first subband.

[0314] Optionally, the second determining submodule is configured to include:

[0315] determining, based on the number of subbands included in the first candidate subband set and the subband bandwidth of each subband, a smoothing weight coefficient for each subband included in the first candidate subband set;

[0316] Adding the smoothing weight coefficients of the subbands included in the first candidate subband set to obtain a total smoothing weight coefficient of the first candidate subband set;

[0317] multiplying the scale difference value of each subband included in the first candidate subband set by the smoothing weight coefficient to obtain a weighted scale difference value of each subband included in the first candidate subband set;

[0318] adding the weighted scale difference values ​​of the subbands included in the first candidate subband set to obtain a summed scale value of the first candidate subband set;

[0319] The summed scale value of the first candidate subband set is divided by the total smoothing weight coefficient to obtain a total scale value of the first candidate subband set.

[0320] Optionally, the apparatus 1100 further includes:

[0321] a bandwidth detection module, configured to perform bandwidth detection on the spectrum of the audio signal to obtain a cutoff frequency of the audio signal if the encoding rate of the audio signal is less than a first rate threshold;

[0322] The second determining module is configured to determine the cutoff sub-bands corresponding to the plurality of sub-band division modes based on the cutoff frequency.

[0323] Optionally, the apparatus 1100 further includes:

[0324] The third determining module is configured to determine the last subband indicated by each of the multiple subband division modes as a cutoff subband corresponding to each subband division mode if the encoding bit rate of the audio signal is not less than the first bit rate threshold.

[0325] Optionally, the apparatus 1100 further includes:

[0326] A feature analysis module is used to perform feature analysis on the frequency spectrum of the audio signal to obtain feature analysis results;

[0327] The fourth determining module is configured to determine a plurality of sub-band division modes from a plurality of candidate sub-band division modes based on the feature analysis result and the coding bit rate of the audio signal.

[0328] Optionally, the feature analysis result includes a subjective signal flag or an objective signal flag, the subjective signal flag indicates that the energy concentration of the audio signal is not greater than the concentration threshold, and the objective signal flag indicates that the energy concentration of the audio signal is greater than the concentration threshold.

[0329] Optionally, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0330] The fourth determination module includes:

[0331] a third determining submodule, configured to determine a first group of sub-band division modes from the plurality of candidate sub-band division modes as the plurality of sub-band division modes if the encoding bit rate of the audio signal is less than a first bit rate threshold and the feature analysis result includes a subjective signal flag;

[0332] The first group of subbands is divided as follows: { {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142, 160,178,196,217,238,259,280,480}, {0,1,2,3,5,7,9,12,15,18,22,26,30,35,41,48,56,65,74,84,94,106,118,134,150, 166,184,202,220,240,260,280,480}, {0,1,2,3,4,5,7,9,11,14,17,21,25,29,34,40,46,52,60,68,76,86,98,110,126,144, 162,180,200,224,250,280,480}, {0,2,4,6,8,12,16,21,26,31,36,41,46,51,56,61,66,71,77,83,89,95,103,111,121, 131,147,163,179,203,240,280,480}, {0,1,2,3,5,7,9,12,15,19,23,27,32,37,43,49,57,66,76,86,98,110,125,140,158, 176,194,216,238,264,290,320,480}, {0,1,2,3,5,7,10,13,17,21,25,30,35,41,47,54,62,70,80,90,102,114,130,146,162, 180,198,218,240,264,290,320,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,66,76,88,100,112,128,144,160,182, 204,226,256,286,316,352,400,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,68,78,90,102,116,132,148,166,186, 208,234,262,292,324,360,400,480}}.

[0333] Optionally, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0334] The fourth determination module includes:

[0335] a fourth determining submodule, configured to determine a second group of sub-band division schemes from the plurality of candidate sub-band division schemes as the plurality of sub-band division schemes if the encoding bit rate of the audio signal is not less than the first bit rate threshold and / or the feature analysis result includes an objective signal flag;

[0336] The second group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,26,30,35,40,45,50,57,64,73,82,92,102, 112,124,136,148,160,480}, {0,1,2,3,4,5,7,9,11,13,15,18,21,24,28,33,38,44,50,57,64,73,82,93,104,116, 128,140,155,170,185,200,480}, {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142, 160,178,196,217,238,259,280,480}, {0,1,2,4,6,10,14,18,22,26,30,34,42,50,58,66,74,84,96,108,120,136,152,168, 192,216,240,272,304,336,376,424,480}, {0,1,2,4,6,10,14,18,26,34,42,50,62,74,86,98,112,128,144,160,176,196,216,236, 256,280,304,328,352,384,416,448,480}, {0,80,92,104,112,120,128,136,144,148,152,156,160,164,168,172,176,180,184, 188,192,196,200,208,216,224,232,240,248,256,268,280,480}, {0,200,212,224,232,240,248,256,264,268,272,276,280,284,288,292,296,300,304, 308,312,316,320,328,336,344,352,360,368,376,388,400,480}, {0,320,332,344,356,364,372,380,384,388,392,396,400,404,408,412,416,420,424, 428,432,436,440,444,448,452,456,460,464,468,472,476,480}}.

[0337] Optionally, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0338] The fourth determination module includes:

[0339] a fifth determining submodule, configured to determine a third group of sub-band division methods from the plurality of candidate sub-band division methods as the plurality of sub-band division methods if the encoding bit rate of the audio signal is less than the first bit rate threshold and the feature analysis result includes a subjective signal flag;

[0340] The third group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,12,14,16,19,22,26,30,35,39,44,50,56,63,71,80,89, 98,108,119,129,140,240}, {0,1,2,3,4,5,6,7,8,9,11,13,15,17,20,24,28,32,37,42,47,53,59,67,75,83,92, 101,110,120,130,140,240}, {0,1,2,3,4,5,6,7,8,9,10,11,12,14,17,20,23,26,30,34,38,43,49,55,63,72,81,90, 100,112,125,140,240}, {0,1,2,3,4,6,8,10,13,15,18,20,23,25,28,30,33,35,38,41,44,47,51,55,60,65,73, 81,89,101,120,140,240}, {0,1,2,3,4,5,6,7,8,10,12,14,17,20,23,27,31,35,40,45,51,57,65,73,81,90,99, 109,120,132,145,160,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,33,38,44,50,56,64,72,80,91,102,113, 128,143,158,176,200,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,34,39,45,51,58,66,74,83,93,104,117, 131,146,162,180,200,240}}.

[0341] Optionally, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz;

[0342] The fourth determination module includes:

[0343] a sixth determining submodule, configured to determine a fourth group of sub-band division schemes from the plurality of candidate sub-band division schemes as the plurality of sub-band division schemes if the encoding bit rate of the audio signal is not less than the first bit rate threshold and / or the feature analysis result includes an objective signal flag;

[0344] The fourth group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,30, 32,34,37,40,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20,22,24,26,28,30,32,34,36,38, 41,44,47,50,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,17,19,21,24,27,30,34,38,42,48,54,60,68, 76,84,94,106,120}, {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,25,28,32,36,40,44,49,54,59,64,70,76,82, 88,96,104,112,120}, {0,20,23,26,28,30,32,34,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,52,54, 56,58,60,62,64,67,70,120}, ,98,99,100,101,102,103,104,105,106,107,108,109, 110,111,112,113,114,115,116,117,118,119,120}}.

[0345] Optionally, the audio signal is a two-channel signal;

[0346] The apparatus 1100 further includes:

[0347] a fifth determining module, configured to determine a first total scaling value based on the scaling factors and subband bandwidths of the subbands included in the target subband set;

[0348] A transformation module, configured to perform addition and subtraction stereo transformation on the spectrum of the two-channel signal to obtain the spectrum of the transformed two-channel signal;

[0349] a sixth determining module, configured to determine a scale factor after the transformation of each subband in the target subband set based on a spectrum value of the transformed binaural signal in each subband included in the target subband set;

[0350] a seventh determining module, configured to determine a second total scaling value based on the transformed scaling factors and subband bandwidths of the subbands included in the target subband set;

[0351] An eighth determining module is configured to determine the two-channel signal as a signal to be encoded if the first total scale value is not greater than the second total scale value.

[0352] Optionally, the apparatus 1100 is further configured to:

[0353] If the first total scale value is greater than the second total scale value, and the encoding bit rate of the audio signal is not less than the first bit rate threshold, and / or the energy concentration of the audio signal is greater than the concentration threshold, the transformed two-channel signal is determined as the signal to be encoded.

[0354] Optionally, the scale factor includes a left channel scale factor and a right channel scale factor;

[0355] The apparatus 1100 is further configured to:

[0356] a ninth determining module, configured to determine, if the first total scale value is greater than the second total scale value, the encoding bit rate of the audio signal is less than the first bit rate threshold, and the energy concentration of the audio signal is not greater than the concentration threshold, a left-right scale factor difference value for each subband included in the target subband set based on the left channel scale factor and the right channel scale factor of each subband included in the target subband set;

[0357] a tenth determining module, configured to determine a subband center frequency of each subband included in the target subband set based on an initial frequency point and a cutoff frequency point of each subband included in the target subband set;

[0358] The eleventh determining module is configured to determine the binaural signal as a signal to be encoded if there is at least one subband in the target subband set whose left and right scale factor difference value is greater than a difference threshold and whose subband center frequency is within a first range.

[0359] Optionally, the apparatus 1100 is further configured to:

[0360] If at least one subband does not exist in the target subband set, the transformed binaural signal is determined as the signal to be encoded.

[0361] In an embodiment of the present application, the optimal sub-band division method is selected from multiple sub-band division methods based on the characteristics of the audio signal. That is, the sub-band division method has the characteristics of signal adaptation and can adapt to the encoding bit rate of the audio signal, thereby improving anti-interference capabilities. Specifically, the audio signal is first divided according to multiple sub-band division methods. Then, based on the spectral values ​​of the audio signal in each divided sub-band, the bandwidth of each sub-band, and the encoding bit rate of the audio signal, the total scale value corresponding to each sub-band division method is determined. Based on the total scale value, the optimal target sub-band division method is selected, that is, the optimal sub-band set is obtained. Subsequent spectrum envelope shaping according to the scaling factors of each sub-band in the optimal sub-band set can improve the encoding effect and compression efficiency.

[0362] It should be noted that the audio signal processing device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate the processing of audio signals. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the audio signal processing device provided in the above embodiment and the audio signal processing method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and is not repeated here.

[0363] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0364] It should be understood that the "at least one" mentioned herein refers to one or more, and "a plurality of" refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0365] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the audio signals involved in the embodiments of this application are all obtained with full authorization.

[0366] The above description is an embodiment provided for this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A method for processing an audio signal, characterized in that: The method comprises: According to multiple sub-band division modes and cutoff sub-bands corresponding to the multiple sub-band division modes, the audio signal is respectively divided into sub-bands to obtain multiple candidate sub-band sets, wherein the multiple candidate sub-band sets correspond to the multiple sub-band division modes one by one, and each candidate sub-band set includes multiple sub-bands; Determine a total scale value of each candidate subband set based on a spectrum value of the audio signal in a subband included in each candidate subband set, a coding rate of the audio signal, and a subband bandwidth of the subband included in each candidate subband set; According to the total scale value of each candidate subband set, a candidate subband set is selected from the multiple candidate subband sets as a target subband set, each subband included in the target subband set has a scale factor, and the scale factor is used to shape the spectrum envelope of the audio signal.

2. The method according to claim 1, characterized in that The step of selecting a candidate subband set as a target subband set from the plurality of candidate subband sets according to the total scale values ​​of the candidate subband sets comprises: A candidate subband set with the smallest total scale value among the multiple candidate subband sets is determined as the target subband set.

3. The method according to claim 1 or 2, characterized in that The determining, based on the spectrum value of the audio signal in the subbands included in each candidate subband set, the encoding bit rate of the audio signal, and the subband bandwidth of the subbands included in each candidate subband set, comprises: For a first candidate subband set among the multiple candidate subband sets, determining a scaling factor of each subband included in the first candidate subband set based on a spectrum value of the audio signal in each subband included in the first candidate subband set, wherein the first candidate subband set is any candidate subband set among the multiple candidate subband sets; Based on the coding rate of the audio signal, and the scale factor and subband bandwidth of each subband included in the first candidate subband set, a total scale value of the first candidate subband set is determined.

4. The method according to claim 3, characterized in that The determining, based on the spectrum value of the audio signal in each subband included in the first candidate subband set, a scaling factor of each subband included in the first candidate subband set comprises: For a first subband included in the first candidate subband set, obtaining a maximum value of absolute values ​​of all frequency spectrum values ​​of the audio signal in the first subband, where the first subband is any subband in the first candidate subband set; Based on the maximum value, a scale factor for the first subband is determined.

5. The method according to claim 3 or 4, characterized in that The encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold; The determining, based on the coding rate of the audio signal and the scaling factor and subband bandwidth of each subband included in the first candidate subband set, a total scaling value of the first candidate subband set comprises: Determining an energy smoothing reference value based on the encoding bit rate of the audio signal and a second bit rate threshold; Based on the energy smoothing reference value, the scaling factors of the subbands included in the first candidate subband set, and the subband band width, determining a total energy value of each subband included in the first candidate subband set; Total energy values ​​of the subbands included in the first candidate subband set are added together to obtain a total scale value of the first candidate subband set.

6. The method according to claim 5, characterized in that The determining, based on the energy smoothing reference value, the scaling factor and the subband bandwidth of each subband included in the first candidate subband set, a total energy value of each subband included in the first candidate subband set comprises: For a first subband included in the first candidate subband set, determining a maximum value between a scale factor of the first subband and the energy smoothing reference value as a reference scale value of the first subband, wherein the first subband is any subband in the first candidate subband set; The product of the reference scale value of the first sub-band and the sub-band bandwidth of the first sub-band is determined as the total energy value of the first sub-band.

7. The method according to claim 3 or 4, characterized in that The encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold; The determining, based on the coding rate of the audio signal and the scaling factor and subband bandwidth of each subband included in the first candidate subband set, a total scaling value of the first candidate subband set comprises: Determining an energy smoothing reference value based on the encoding bit rate of the audio signal and a second bit rate threshold; Determine, based on the energy smoothing reference value and the scale factor of each subband included in the first candidate subband set, a scale difference value of each subband included in the first candidate subband set, wherein the scale difference value represents a difference between a scale factor of a corresponding subband and a scale factor of an adjacent subband of the corresponding subband; Based on the scale difference value and the subband bandwidth of each subband included in the first candidate subband set, a total scale value of the first candidate subband set is determined.

8. The method according to claim 7, characterized in that The determining, based on the energy smoothing reference value and the scaling factor of each subband included in the first candidate subband set, a scaling difference value of each subband included in the first candidate subband set comprises: For a first subband included in the first candidate subband set, determining a first smoothing value, a second smoothing value, and a third smoothing value of the first subband based on the energy smoothing reference value, a scale factor of the first subband, and a scale factor of an adjacent subband of the first subband, wherein the first subband is any subband in the first candidate subband set; A scaled difference value for the first subband is determined based on the first smoothed value, the second smoothed value, and the third smoothed value of the first subband.

9. The method according to claim 8, characterized in that The determining, based on the energy smoothing reference value, the scaling factor of the first subband and the scaling factors of the adjacent subbands of the first subband, a first smoothing value, a second smoothing value and a third smoothing value of the first subband comprises: If the first subband is the first subband in the first candidate subband set, the maximum value of the scaling factor of the first subband and the energy smoothing reference value is determined as the first smoothing value of the first subband; if the first subband is not the first subband in the first candidate subband set, the scaling factor of the previous adjacent subband of the first subband is and the energy smoothing reference value, determining a maximum value of the first sub-band as a first smoothing value of the first sub-band; Determine the maximum value of the scaling factor of the first sub-band and the energy smoothing reference value as the second smoothing value of the first sub-band; If the first subband is the last subband in the first candidate subband set, the maximum value of the scaling factor of the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband; if the first subband is not the last subband in the first candidate subband set, the maximum value of the scaling factor of the next adjacent subband of the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband.

10. The method according to claim 8 or 9, characterized in that The determining the scale difference value of the first sub-band based on the first smoothed value, the second smoothed value and the third smoothed value of the first sub-band comprises: For a first subband included in the first candidate subband set, determining a first difference value and a second difference value of the first subband, wherein the first difference value refers to an absolute value of a difference between a first smoothed value and a second smoothed value of the first subband, and the second difference value refers to an absolute value of a difference between a second smoothed value and a third smoothed value of the first subband, and the first subband is any subband in the first candidate subband set; A scaled difference value for the first subband is determined based on the first difference value and the second difference value for the first subband.

11. The method according to any one of claims 7 to 10, characterized in that: The determining, based on the scale difference value and the subband bandwidth of each subband included in the first candidate subband set, a total scale value of the first candidate subband set comprises: Determining a smoothing weighting coefficient of each subband included in the first candidate subband set based on the number of subbands included in the first candidate subband set and the subband bandwidth of each subband; adding the smoothing weighting coefficients of the subbands included in the first candidate subband set to obtain a total smoothing weighting coefficient of the first candidate subband set; multiplying the scale difference value of each subband included in the first candidate subband set by a smoothing weight coefficient to obtain a weighted scale difference value of each subband included in the first candidate subband set; adding weighted scaled difference values ​​of the subbands included in the first candidate subband set to obtain a summed scaled value of the first candidate subband set; The summed scaled value of the first candidate subband set is divided by the total smoothing weight coefficient to obtain a total scaled value of the first candidate subband set.

12. The method according to any one of claims 1 to 11, characterized in that: The method further comprises: If the encoding bit rate of the audio signal is less than a first bit rate threshold, performing bandwidth detection on the frequency spectrum of the audio signal to obtain a cutoff frequency of the audio signal; Based on the cutoff frequency, cutoff sub-bands corresponding to the multiple sub-band division modes are determined.

13. The method according to any one of claims 1 to 12, characterized in that: The method further comprises: If the encoding bit rate of the audio signal is not less than the first bit rate threshold, the last subband indicated by each subband division method among the multiple subband division methods is determined as a cutoff subband corresponding to each subband division method.

14. The method according to any one of claims 1 to 13, characterized in that: The method further comprises: Performing feature analysis on the frequency spectrum of the audio signal to obtain a feature analysis result; The multiple sub-band division modes are determined from multiple candidate sub-band division modes based on the feature analysis result and the encoding bit rate of the audio signal.

15. The method according to claim 14, characterized in that The feature analysis result includes a subjective signal flag or an objective signal flag, the subjective signal flag indicates that the energy concentration of the audio signal is not greater than a concentration threshold, and the objective signal flag indicates that the energy concentration of the audio signal is greater than the concentration threshold.

16. The method according to claim 15, characterized in that The frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The determining the multiple sub-band division modes from multiple candidate sub-band division modes based on the feature analysis result and the encoding bit rate of the audio signal comprises: If the encoding bit rate of the audio signal is less than a first bit rate threshold, and the feature analysis result includes the subjective signal flag, determining a first group of sub-band division modes among the multiple candidate sub-band division modes as the multiple sub-band division modes; The first group of subbands is divided as follows: { {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142,160,178,196,217,238,259,280,480}, {0,1,2,3,5,7,9,12,15,18,22,26,30,35,41,48,56,65,74,84,94,106,118,134,150,166,184,202,220,240,260,280,480}, {0,1,2,3,4,5,7,9,11,14,17,21,25,29,34,40,46,52,60,68,76,86,98,110,126,144,162,180,200,224,250,280,480}, {0,2,4,6,8,12,16,21,26,31,36,41,46,51,56,61,66,71,77,83,89,95,103,111,121,131,147,163,179,203,240,280,480}, {0,1,2,3,5,7,9,12,15,19,23,27,32,37,43,49,57,66,76,86,98,110,125,140,158,176,194,216,238,264,290,320,480}, {0,1,2,3,5,7,10,13,17,21,25,30,35,41,47,54,62,70,80,90,102,114,130,146,162,180,198,218,240,264,290,320,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,66,76,88,100,112,128,144,160,182,204,226,256,286,316,352,400,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,68,78,90,102,116,132,148,166,186,208,234,262,292,324,360,400,480} }。 17. The method according to claim 15, characterized in that The frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The determining the multiple sub-band division modes from multiple candidate sub-band division modes based on the feature analysis result and the bit rate of the audio signal includes: If the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the feature analysis result includes the objective signal flag, determining a second group of sub-band division modes among the multiple candidate sub-band division modes as the multiple sub-band division modes; The second group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,26,30,35,40,45,50,57,64,73,82,92,102,112,124,136,148,160,480}, {0,1,2,3,4,5,7,9,11,13,15,18,21,24,28,33,38,44,50,57,64,73,82,93,104,116,128,140,155,170,185,200,480}, {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142,160,178,196,217,238,259,280,480}, {0,1,2,4,6,10,14,18,22,26,30,34,42,50,58,66,74,84,96,108,120,136,152,168,192,216,240,272,304,336,376,424,480}, {0,1,2,4,6,10,14,18,26,34,42,50,62,74,86,98,112,128,144,160,176,196,216,236,256,280,304,328,352,384,416,448,480}, {0,80,92,104,112,120,128,136,144,148,152,156,160,164,168,172,176,180,184,188,192,196,200,208,216,224,232,240,248,256,268,280,480}, {0,200,212,224,232,240,248,256,264,268,272,276,280,284,288,292,296,300,304,308,312,316,320,328,336,344,352,360,368,376,388,400,480}, {0,320,332,344,356,364,372,380,384,388,392,396,400,404,408,412,416,420,424,428,432,436,440,444,448,452,456,460,464,468,472,476,480} }。 18. The method according to claim 15, characterized in that The frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The determining the multiple sub-band division modes from multiple candidate sub-band division modes based on the feature analysis result and the bit rate of the audio signal includes: If the encoding bit rate of the audio signal is less than a first bit rate threshold, and the feature analysis result includes the subjective signal flag, determining a third group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods; The third group of sub-bands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,12,14,16,19,22,26,30,35,39,44,50，56,63,71，80，89，98，108,119,129,140,240}, {0,1,2,3,4,5,6,7,8,9,11,13,15，17,20,24,28,32,37,42,47,53,59,67,75,83，92,101,110,120,130,140,240}, {0,1,2,3,4,5,6,7,8，9,10,11,12,14,17,20,23,26,30,34,38,43,49,55,63,72,81,90, 100,112,125,140,240}, {0,1,2,3,4,6,8,10,13,15,18,20,23,25,28,30,33,35，38,41,44,47,51,55,60,65,73,81，89,101,120,140,240}, {0,1,2,3,4,5,6,7,9,11,13,14,16,18,21,24,28,33,38,43,49,55,62,70,79,88,97,108,119,132,145,160,240}, {0,1,2,3,4,5,6,7,8,10,12,14,17,20,23,27,31,35,40,45,51,57,65,73,81,90,99,109,120,132,145,160,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,33,38,44,50,56,64,72,80,91,102,113,128,143,158,176,200,240}, {0,1，2,3,4,5,6,7,9,11,13,15,18,21,25,29,34,39,45,51,58,66,74,83,93,104,117,131,146,162,180,200,240} }。 19. The method according to claim 15, characterized in that The frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The determining the multiple sub-band division modes from multiple candidate sub-band division modes based on the feature analysis result and the bit rate of the audio signal includes: If the encoding bit rate of the audio signal is not less than the first bit rate threshold, and / or the feature analysis result includes the objective signal flag, determining a fourth group of sub-band division methods among the multiple candidate sub-band division methods as the multiple sub-band division methods; The fourth group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,30,32,34,37,40,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20,22,24,26,28,30,32,34,36,38,41,44,47,50,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,14,16,18,20,22,24,26,28,31,34,37,40,44,48,52,56,60,65,70,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,17,19,21,24,27,30,34,38,42,48,54,60,68,76,84,94,106,120}, {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,25,28,32,36,40,44,49,54,59,64,70,76,82,88,96,104,112,120}, {0,20,23,26,28,30,32,34,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,52,54,56,58,60,62,64,67,70,120}, {0,50,53,56,58,60,62,64,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,82,84,86,88,90,92,94,97,100,120}, {0,80,83,86,89,91,93,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120} }。 20. The method according to any one of claims 1 to 19, characterized in that: The audio signal is a dual-channel signal; The method further comprises: Determining a first total scaling value based on a scaling factor and a subband bandwidth of each subband included in the target subband set; Performing addition and subtraction stereo transformation on the frequency spectrum of the two-channel signal to obtain a transformed frequency spectrum of the two-channel signal; Determining a transformed scaling factor of each subband in the target subband set based on a spectrum value of the transformed binaural signal in each subband included in the target subband set; Determining a second total scaling value based on the transformed scaling factors and subband bandwidths of the subbands included in the target subband set; If the first total scale value is not greater than the second total scale value, the two-channel signal is determined as a signal to be encoded.

21. The method of claim 20, wherein: The method further comprises: If the first total scale value is greater than the second total scale value, and the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold, the transformed dual-channel signal is determined as the signal to be encoded.

22. The method according to claim 20 or 21, characterized in that The scaling factors include a left channel scaling factor and a right channel scaling factor; The method further comprises: If the first total scale value is greater than the second total scale value, the encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold, determining a left-right scale factor difference value of each subband included in the target subband set based on a left channel scale factor and a right channel scale factor of each subband included in the target subband set; Determining a subband center frequency of each subband included in the target subband set based on an initial frequency point and a cutoff frequency point of each subband included in the target subband set; If there is at least one subband in the target subband set whose left-right scale factor difference value is greater than the difference threshold and the subband center frequency is within a first range, the binaural signal is determined as a signal to be encoded.

23. The method of claim 22, wherein: The method further comprises: If the at least one subband does not exist in the target subband set, the transformed binaural signal is determined as the signal to be encoded.

24. An audio signal processing device, characterized in that: The device comprises: a subband division module, configured to perform subband division on the audio signal according to multiple subband division modes and cutoff subbands corresponding to the multiple subband division modes, respectively, to obtain multiple candidate subband sets, wherein the multiple candidate subband sets correspond to the multiple subband division modes one by one, and each candidate subband set includes multiple subbands; The first determining module is configured to determine, based on the spectrum value of the audio signal in the subbands included in each candidate subband set, the encoding bit rate of the audio signal, and the subband bandwidth of the subbands included in each candidate subband set, The total scale value of the collection; The selection module is used to select a candidate subband set from the multiple candidate subband sets as a target subband set according to the total scale value of each candidate subband set, each subband included in the target subband set has a scale factor, and the scale factor is used to shape the spectrum envelope of the audio signal.

25. The device according to claim 24, characterized in that The selection module is used to: A candidate subband set with the smallest total scale value among the multiple candidate subband sets is determined as the target subband set.

26. The device according to claim 24 or 25, characterized in that The first determining module includes: a first determining submodule, configured to determine, for a first candidate subband set among the multiple candidate subband sets, a scaling factor of each subband included in the first candidate subband set based on a spectrum value of the audio signal in each subband included in the first candidate subband set, wherein the first candidate subband set is any candidate subband set among the multiple candidate subband sets; The second determination submodule is configured to determine a total scaling value of the first candidate subband set based on a coding rate of the audio signal and a scaling factor and a subband bandwidth of each subband included in the first candidate subband set.

27. The device according to claim 26, characterized in that The second determining submodule is used for: For a first subband included in the first candidate subband set, obtaining a maximum value of absolute values ​​of all frequency spectrum values ​​of the audio signal in the first subband, where the first subband is any subband in the first candidate subband set; Based on the maximum value, a scale factor for the first subband is determined.

28. The device according to claim 26 or 27, characterized in that The encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold; The second determining submodule is used for: Determining an energy smoothing reference value based on the encoding bit rate of the audio signal and a second bit rate threshold; Determine a total energy value of each subband included in the first candidate subband set based on the energy smoothing reference value, the scaling factor and the subband bandwidth of each subband included in the first candidate subband set; Total energy values ​​of the subbands included in the first candidate subband set are added together to obtain a total scale value of the first candidate subband set.

29. The device according to claim 28, characterized in that The second determining submodule is used for: For a first subband included in the first candidate subband set, determining a maximum value between a scale factor of the first subband and the energy smoothing reference value as a reference scale value of the first subband, wherein the first subband is any subband in the first candidate subband set; The product of the reference scale value of the first sub-band and the sub-band bandwidth of the first sub-band is determined as the total energy value of the first sub-band.

30. The device according to claim 26 or 27, characterized in that The encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold; The second determining submodule is used for: Determining an energy smoothing reference value based on the encoding bit rate of the audio signal and a second bit rate threshold; Determine, based on the energy smoothing reference value and the scale factor of each subband included in the first candidate subband set, a scale difference value of each subband included in the first candidate subband set, wherein the scale difference value represents a difference between a scale factor of a corresponding subband and a scale factor of an adjacent subband of the corresponding subband; Based on the scale difference value and the subband bandwidth of each subband included in the first candidate subband set, a total scale value of the first candidate subband set is determined.

31. The device according to claim 30, characterized in that The second determining submodule is used for: For a first subband included in the first candidate subband set, determining a first smoothing value, a second smoothing value, and a third smoothing value of the first subband based on the energy smoothing reference value, a scale factor of the first subband, and a scale factor of an adjacent subband of the first subband, wherein the first subband is any subband in the first candidate subband set; A scaled difference value for the first subband is determined based on the first smoothed value, the second smoothed value, and the third smoothed value of the first subband.

32. The device according to claim 31, characterized in that The second determining submodule is used for: If the first subband is the first subband in the first candidate subband set, a maximum value between a scale factor of the first subband and the energy smoothing reference value is determined as a first smoothing value of the first subband; if the first subband is not the first subband in the first candidate subband set, a maximum value between a scale factor of a previous adjacent subband of the first subband and the energy smoothing reference value is determined as the first smoothing value of the first subband; Determine the maximum value of the scaling factor of the first sub-band and the energy smoothing reference value as the second smoothing value of the first sub-band; If the first subband is the last subband in the first candidate subband set, the maximum value of the scaling factor of the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband; if the first subband is not the last subband in the first candidate subband set, the maximum value of the scaling factor of the next adjacent subband of the first subband and the energy smoothing reference value is determined as the third smoothing value of the first subband.

33. The device according to claim 31 or 32, characterized in that The second determining submodule is used for: For a first subband included in the first candidate subband set, determining a first difference value and a second difference value of the first subband, wherein the first difference value refers to an absolute value of a difference between a first smoothed value and a second smoothed value of the first subband, and the second difference value refers to an absolute value of a difference between a second smoothed value and a third smoothed value of the first subband, and the first subband is any subband in the first candidate subband set; A scaled difference value for the first subband is determined based on the first difference value and the second difference value for the first subband.

34. The device according to any one of claims 30 to 33, characterized in that: The second determining submodule is used to include: Determining a smoothing weighting coefficient of each subband included in the first candidate subband set based on the number of subbands included in the first candidate subband set and the subband bandwidth of each subband; adding the smoothing weighting coefficients of the subbands included in the first candidate subband set to obtain a total smoothing weighting coefficient of the first candidate subband set; multiplying the scale difference value of each subband included in the first candidate subband set by a smoothing weight coefficient to obtain a weighted scale difference value of each subband included in the first candidate subband set; adding weighted scaled difference values ​​of the subbands included in the first candidate subband set to obtain a summed scaled value of the first candidate subband set; The summed scaled value of the first candidate subband set is divided by the total smoothing weight coefficient to obtain a total scaled value of the first candidate subband set.

35. The device according to any one of claims 24 to 34, characterized in that: The device also includes: a bandwidth detection module, configured to perform bandwidth detection on the frequency spectrum of the audio signal to obtain a cutoff frequency of the audio signal if the encoding bit rate of the audio signal is less than a first bit rate threshold; The second determination module is configured to determine the cutoff sub-bands corresponding to the multiple sub-band division modes respectively based on the cutoff frequency.

36. The device according to any one of claims 24 to 35, characterized in that: The device also includes: The third determination module is configured to determine the last subband indicated by each subband division method among the multiple subband division methods as a cutoff subband corresponding to each subband division method if the encoding bit rate of the audio signal is not less than the first bit rate threshold.

37. The device according to any one of claims 24 to 36, characterized in that: The device also includes: A feature analysis module, used to perform feature analysis on the frequency spectrum of the audio signal to obtain a feature analysis result; The fourth determination module is used to determine the multiple sub-band division modes from multiple candidate sub-band division modes based on the feature analysis result and the encoding bit rate of the audio signal.

38. The device according to claim 37, characterized in that The feature analysis result includes a subjective signal flag or an objective signal flag, the subjective signal flag indicates that the energy concentration of the audio signal is not greater than a concentration threshold, and the objective signal flag indicates that the energy concentration of the audio signal is greater than the concentration threshold.

39. The device according to claim 38, characterized in that The frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The fourth determination module comprises: a third determination submodule, configured to determine a first group of sub-band division modes among the multiple candidate sub-band division modes as the multiple sub-band division modes if the encoding bit rate of the audio signal is less than a first bit rate threshold and the feature analysis result includes the subjective signal flag; The first group of subbands is divided as follows: { {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142,160,178,196,217,238,259,280,480}, {0,1,2,3,5,7,9,12,15,18,22,26,30,35,41,48,56,65,74,84,94,106,118,134,150, 166,184,202,220,240,260,280,480}, {0,1,2,3,4,5,7,9,11,14,17,21,25,29,34,40,46,52,60,68,76,86,98,110,126,144,162,180,200,224,250,280,480}, {0,2,4,6,8,12,16,21,26,31,36,41,46,51,56,61,66,71,77,83,89,95,103,111,121,131,147,163,179,203,240,280,480}, {0,1,2,3,5,7,9,12,15,19,23,27,32,37,43,49,57,66,76,86,98,110,125,140,158,176,194,216,238,264,290,320,480}, {0,1,2,3,5,7,10,13,17,21,25,30,35,41,47,54,62,70,80,90,102,114,130,146,162,180,198,218,240,264,290,320,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,66,76,88,100,112,128,144,160,182,204,226,256,286,316,352,400,480}, {0,1,2,4,6,8,11,14,18,22,26,30,36,42,50,58,68,78,90,102,116,132,148,166,186,208,234,262,292,324,360,400,480} }。 40. The device of claim 38, wherein: The frame length of the audio signal is 10 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 5 milliseconds, and the sampling rate is 88.2 kHz or 96 kHz; or, the frame length of the audio signal is 10 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The fourth determination module comprises: a fourth determination submodule, configured to determine a second group of sub-band division modes among the multiple candidate sub-band division modes as the multiple sub-band division modes if the encoding bit rate of the audio signal is not less than a first bit rate threshold and / or the feature analysis result includes the objective signal flag; The second group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,26,30,35,40,45,50,57,64,73,82,92,102,112,124,136,148,160,480}, {0,1,2,3,4,5,7,9,11,13,15,18,21,24,28,33,38,44,50,57,64,73,82,93,104,116,128,140,155,170,185,200,480}, {0,1,2,3,4,6,8,10,13,16,20,24,28,33,38,45,52,61,70,79,88,100,112,127,142,160,178,196,217,238,259,280,480}, {0,1,2,4,6,10,14,18,22,26,30,34,42,50,58,66,74,84,96,108,120,136,152,168,192,216,240,272,304,336,376,424,480}, {0,1,2,4,6,10,14,18,26,34,42,50,62,74,86,98,112,128,144,160,176,196,216,236,256,280,304,328,352,384,416,448,480}, {0,80,92,104,112,120,128,136,144,148,152,156,160,164,168,172,176,180,184,188,192,196,200,208,216,224,232,240,248,256,268,280,480}, {0,200,212,224,232,240,248,256,264,268,272,276,280,284,288,292,296,300,304,308,312,316,320,328,336,344,352,360,368,376,388,400,480}, {0,320,332,344,356,364,372,380,384,388,392,396,400,404,408,412,416,420,424, 428,432,436,440,444,448,452,456,460,464,468,472,476,480} }。 41. The device according to claim 38, characterized in that The frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The fourth determination module comprises: a fifth determination submodule, configured to determine a third group of sub-band division modes among the multiple candidate sub-band division modes as the multiple sub-band division modes if the encoding bit rate of the audio signal is less than a first bit rate threshold and the feature analysis result includes the subjective signal flag; The third group of sub-bands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,12,14,16,19,22,26,30,35,39,44,50，56,63,71，80，89，98，108,119,129,140,240}, {0,1,2,3,4,5,6,7,8,9,11,13,15，17,20,24,28,32,37,42,47,53,59,67,75,83，92,101,110,120,130,140,240}, {0,1,2,3,4,5,6,7,8，9,10,11,12,14,17,20,23,26,30,34,38,43,49,55,63,72,81,90,100,112,125,140,240}, {0,1,2,3,4,6,8,10,13,15,18,20,23,25,28,30,33,35，38,41,44,47,51,55,60,65,73,81，89,101,120,140,240}, {0,1,2,3,4,5,6,7,9,11,13,14,16,18,21,24,28,33,38,43,49,55,62,70,79,88,97,108,119,132,145,160,240}, {0,1,2,3,4,5,6,7,8,10,12,14,17,20,23,27,31,35,40,45,51,57,65,73,81,90,99,109,120,132,145,160,240}, {0,1,2,3,4,5,6,7,9,11,13,15,18,21,25,29,33,38,44,50,56,64,72,80,91,102,113,128,143,158,176,200,240}, {0,1，2,3,4,5,6,7,9,11,13,15,18,21,25,29,34,39,45,51,58,66,74,83,93,104,117,131,146,162,180,200,240} }。 42. The device according to claim 38, characterized in that The frame length of the audio signal is 5 milliseconds, and the sampling rate is 44.1 kHz or 48 kHz; The fourth determination module comprises: a sixth determination submodule, configured to determine a fourth group of sub-band division modes among the multiple candidate sub-band division modes as the multiple sub-band division modes if the encoding bit rate of the audio signal is not less than a first bit rate threshold and / or the feature analysis result includes the objective signal flag; The fourth group of subbands is divided as follows: { {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,30,32,34,37,40,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20,22,24,26,28,30,32,34,36,38,41,44,47,50,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,14,16,18,20,22,24,26,28,31,34,37,40,44,48,52,56,60,65,70,120}, {0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,17,19,21,24,27,30,34,38,42,48,54,60,68,76,84,94,106,120}, {0,1,2,3,4,5,6,7,8,10,12,14,16,19,22,25,28,32,36,40,44,49,54,59,64,70,76,82,88,96,104,112,120}, {0,20,23,26,28,30,32,34,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,52,54,56,58,60,62,64,67,70,120}, {0,50,53,56,58,60,62,64,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,82,84,86,88,90,92,94,97,100,120}, {0,80,83,86,89,91,93,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120} }。 43. The device according to any one of claims 24 to 42, characterized in that: The audio signal is a dual-channel signal; The device also includes: a fifth determination module, configured to determine a first total scaling value based on a scaling factor and a subband bandwidth of each subband included in the target subband set; A transformation module, used for performing addition and subtraction stereo transformation on the spectrum of the two-channel signal to obtain the spectrum of the transformed two-channel signal; a sixth determination module, configured to determine a transformed scaling factor of each subband in the target subband set based on a spectrum value of the transformed binaural signal in each subband included in the target subband set; a seventh determination module, configured to determine a second total scale value based on the transformed scale factors and subband bandwidths of the subbands included in the target subband set; An eighth determination module is configured to determine the two-channel signal as a signal to be encoded if the first total scale value is not greater than the second total scale value.

44. The device according to claim 43, characterized in that The device is also used for: If the first total scale value is greater than the second total scale value, and the encoding bit rate of the audio signal is not less than a first bit rate threshold, and / or the energy concentration of the audio signal is greater than a concentration threshold, the transformed dual-channel signal is determined as the signal to be encoded.

45. The device according to claim 43 or 44, characterized in that The scaling factors include a left channel scaling factor and a right channel scaling factor; The device also includes: a ninth determining module, configured to determine, based on the target, if the first total scale value is greater than the second total scale value, the encoding bit rate of the audio signal is less than a first bit rate threshold, and the energy concentration of the audio signal is not greater than a concentration threshold. The left channel scaling factor and the right channel scaling factor of each subband included in the target subband set are determined, and the difference value of the left and right scaling factors of each subband included in the target subband set is determined; a tenth determining module, configured to determine a subband center frequency of each subband included in the target subband set based on an initial frequency point and a cutoff frequency point of each subband included in the target subband set; The eleventh determination module is configured to determine the binaural signal as a signal to be encoded if there is at least one subband in the target subband set whose left and right scaling factor difference value is greater than a difference threshold and whose subband center frequency is within a first range.

46. ​​The device according to claim 45, characterized in that The device is also used for: If the at least one subband does not exist in the target subband set, the transformed binaural signal is determined as the signal to be encoded.

47. An audio signal processing device, characterized in that: The device comprises a memory and a processor; The memory is used to store a computer program, wherein the computer program includes program instructions; The processor is used to call the computer program to implement the method according to any one of claims 1 to 23.

48. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 23 are implemented.

49. A computer program product, characterized in that The computer program product stores computer instructions, and when the computer instructions are executed by a processor, the steps of any one of the methods of claims 1-23 are implemented.

Images