Audio coding method and apparatus

Abstract

The application provides an audio coding method and device. The audio coding method comprises the following steps: an audio sending device determines a signal type and a use scenario of a current audio frame to be coded; an initial coding mode of the current audio frame is determined according to the signal type of the current audio frame and the use scenario of the current audio frame; a final coding mode of the current audio frame is determined according to the initial coding mode of the current audio frame and a final coding mode of a previous audio frame, and the final coding mode comprises a time-domain coding to frequency-domain coding switching coding mode or a frequency-domain coding to time-domain coding switching coding mode; the current audio frame is coded according to the final coding mode of the current audio frame; an audio receiving device acquires a code stream; the code stream is parsed to acquire code stream data to be decoded and indication information of a decoding mode; and the code stream data is decoded according to the decoding mode. The application has the advantages of high sound quality and adaptive switching of different coding modes according to signal characteristics and coding modes.

Description

TECHNICAL FIELD

[0001] The present application relates to audio coding technology, and in particular to an audio coding method and device. BACKGROUND

[0002] With the wide popularity and use of wireless Bluetooth devices such as true wireless stereo (TWS) earphones, smart sound boxes, and smart watches in people's daily life, people's demand for high-quality music playing experience in various scenarios has become increasingly urgent, especially in environments such as subways, airports, and train stations where Bluetooth signals are easily disturbed. Due to the size limitation of the Bluetooth channel for data transmission, the music data stream must be compressed by the audio encoder of the Bluetooth device sending end before being transmitted to the Bluetooth device receiving end for decoding, which also promotes the vigorous development of various Bluetooth audio codecs.

[0003] Bluetooth audio codecs are mainly applied between Bluetooth interconnected devices (earphones, sound boxes, smart wearable devices, etc.) to provide high-quality music transmission and playback under different scene requirements. Currently, audio coding technology is divided into two types, one is time domain coding technology, for example, linear predictive coding (LPC) based coding technology, which has high coding efficiency for audio signals with obvious spectral envelope such as speech. The other is frequency domain coding technology, for example, modified discrete cosine transform (MDCT) based coding technology, which has high coding efficiency for frequency-rich audio signals such as music.

[0004] Therefore, implementing flexible switching between LPC time domain coding and MDCT frequency domain coding can improve the audio quality after audio coding, which is of great significance for high-definition audio scenarios. SUMMARY

[0005] The present application provides an audio coding method and device, which can adaptively switch between different coding modes according to signal characteristics and coding modes.

[0006] In a first aspect, the present application provides an audio encoding method, comprising: determining a signal type and a use scenario of a current audio frame to be encoded, wherein the use scenario comprises a low-delay scenario or a high-quality scenario; determining an initial encoding mode of the current audio frame according to the signal type of the current audio frame and the use scenario of the current audio frame, wherein the initial encoding mode comprises a time-domain encoding mode or a frequency-domain encoding mode; determining a final encoding mode of the current audio frame according to the initial encoding mode of the current audio frame and a final encoding mode of a previous audio frame, wherein the final encoding mode comprises the time-domain encoding mode, the frequency-domain encoding mode, a time-domain-to-frequency-domain encoding switching mode or a frequency-domain-to-time-domain encoding switching mode; and encoding the current audio frame according to the final encoding mode of the current audio frame.

[0007] The audio frame can be any frame in audio transmitted by an audio sending device to an audio receiving device. In the present application, the object of each encoding can be a frame of audio, i.e., the audio encoding method provided by the present application is for a frame of audio, and the method of determining the encoding mode described below is applicable to each frame of audio. Therefore, in order to distinguish, the audio frame being encoded by the audio sending device is referred to as an audio frame or a current audio frame, and the audio frame being encoded by the audio sending device prior to the audio frame is referred to as a previous audio frame.

[0008] The signal type of the audio frame can comprise a speech type or a music type. The signal type of the audio frame can be determined by attribute information of the audio frame, for example, after a mobile phone reads audio to be transmitted from an audio source file, the attribute information thereof can also be read from the audio source file; or the signal type of the audio frame can be determined by a playing mode of the audio frame, for example, if a user makes a call through a mobile phone, the audio frame transmitted by the mobile phone to a Bluetooth earphone belongs to the speech type, and if the user plays music on the mobile phone by using an application (APP), the audio frame transmitted by the mobile phone to a Bluetooth speaker belongs to the music type. In addition, the audio sending device can obtain the type of the audio frame by other ways, which is not limited in the present application.

[0009] The use scenario of the audio frame can comprise a low-delay scenario or a high-quality scenario. When the frame length of the codec is N, the delay of the time-domain encoding is N sample data or N+D sample data, and the delay of the frequency-domain encoding is 2N sample data or 2N-2M sample data. In the present application, the low-delay scenario by default refers to the time-domain encoding because the delay of the time-domain encoding is lower than that of the frequency-domain encoding. Since the frequency-domain encoding can be combined with a psychoacoustic model, the encoding quality of the time-domain stationary signal is higher than that of the time-domain encoding, and the encoding quality of the non-time-domain stationary signal is higher than that of the time-domain encoding. In the present application, the high-quality mode by default refers to the encoding mode of the fusion of the time-domain and the frequency-domain, which has high-quality encoding effect for both the time-domain stationary signal and the non-time-domain stationary signal.

[0010] The use scenario of the audio frame can be determined by a control or an option clicked by the user on an interactive interface of an APP on the mobile phone, for example, when the user connects Bluetooth on the mobile phone, selects a low-latency scenario or a high-audio-quality scenario on a pop-up interface; or can be determined by the type of the APP opened by the user on the mobile phone, for example, if the APP is a game APP, a live broadcast APP, a phone call APP or a KTV APP, etc., the use scenario is a low-latency scenario; if the APP is an audio player APP, a video player APP, a KTV APP, a recording APP or a broadcast APP, the use scenario is a high-audio-quality scenario. In addition, the audio sending device can obtain the use scenario of the audio frame in other manners, which is not limited in the present application.

[0011] The encoding mode includes a time-domain encoding mode or a frequency-domain encoding mode. When the use scenario of the audio frame is a low-latency scenario, the encoding mode of the audio frame is determined as the time-domain encoding mode. When the use scenario of the audio frame is a high-audio-quality scenario, the audio frame is split into a plurality of audio subframes, the energy variances of the plurality of audio subframes are calculated, it is judged whether the energy variances of the plurality of audio subframes are greater than a first set threshold, when the energy variances of the plurality of audio subframes are greater than the first set threshold, the audio frame is filtered by using LPC encoding to obtain the energy of the filtered audio frame, it is judged whether the ratio of the energy of the filtered audio frame to the energy of the audio frame is greater than a second set threshold, when the ratio of the energy of the filtered audio frame to the energy of the audio frame is greater than the second set threshold, the audio frame is determined as a time-domain stationary signal, when the ratio of the energy of the filtered audio frame to the energy of the audio frame is less than the second set threshold, the audio frame is determined as not a time-domain stationary signal, when the energy variances of the plurality of audio subframes are less than the first set threshold, the audio frame is transformed by using MDCT transformation to obtain the energy variances of each frequency point of the transformed audio frame, it is judged whether the energy variances of each frequency point of the transformed audio frame are greater than a third set threshold, when the energy variances of each frequency point of the transformed audio frame are greater than the third set threshold, the audio frame is determined as a time-domain stationary signal, when the energy variances of each frequency point of the transformed audio frame are less than the third set threshold, the audio frame is determined as not a time-domain stationary signal. When the audio frame is a time-domain stationary signal, the encoding mode of the audio frame is determined as the frequency-domain encoding mode; when the audio frame is not a time-domain stationary signal, the encoding mode of the audio frame is determined as the time-domain encoding mode.

[0012] It should be noted that in the above comparisons with the first set threshold, the second set threshold and the third set threshold, the equal case can be processed as the greater case or the less case, which is not limited in the present application.

[0013] Optionally, when the audio frame is of the speech type, the encoding mode of the audio frame is determined as the time-domain encoding mode. This has been adopted in the related art, and will not be described herein.

[0014] It can be seen that the time-domain encoding mode can be used not only for audio frames of the speech type, but also for audio frames of the music type in a low-delay scenario, and for audio frames of the music type but not time-domain stationary signals in a high-quality scenario. In this way, the low-delay advantage of the time-domain encoding mode, especially the LPC encoding mode, and the high-efficiency encoding advantage of the time-domain encoding mode for audio frames of the speech type and non-stationary signals can be fully exerted.

[0015] The above method determines the encoding mode of the audio frame, and the final encoding mode of the previous audio frame has been determined as one of the above four final encoding modes when it is encoded. Therefore, there are the following cases:

[0016] (1) When the encoding mode of the audio frame and the encoding mode of the previous audio frame are both the frequency-domain encoding mode, the final encoding mode of the audio frame is determined as the frequency-domain encoding mode; or,

[0017] (2) When the encoding mode of the audio frame is the frequency-domain encoding mode, and the encoding mode of the previous audio frame is the time-domain encoding mode, the final encoding mode of the audio frame is determined as the time-domain encoding mode; or,

[0018] (3) When the encoding mode of the audio frame is the frequency-domain encoding mode, and the encoding mode of the previous audio frame is the time-domain encoding mode, the final encoding mode of the audio frame is determined as the frequency-domain encoding mode; or,

[0019] (4) When the encoding mode of the audio frame is the frequency-domain encoding mode, and the encoding mode of the previous audio frame is the frequency-domain encoding mode to time-domain encoding mode switching encoding mode, the final encoding mode of the audio frame is determined as the time-domain encoding mode to frequency-domain encoding mode switching encoding mode; or,

[0020] (5) When the encoding mode of the audio frame and the encoding mode of the previous audio frame are both the time-domain encoding mode, the final encoding mode of the audio frame is determined as the time-domain encoding mode; or,

[0021] (6) When the encoding mode of the audio frame is the time-domain encoding mode, and the encoding mode of the previous audio frame is the frequency-domain encoding mode, the final encoding mode of the audio frame is determined as the frequency-domain encoding mode to time-domain encoding mode switching encoding mode; or,

[0022] (7) When the encoding mode of the audio frame is the time-domain encoding mode, and the encoding mode of the previous audio frame is the time-domain encoding mode to frequency-domain encoding mode switching encoding mode, the final encoding mode of the audio frame is determined as the frequency-domain encoding mode to time-domain encoding mode switching encoding mode; or,

[0023] (8) When the encoding mode of the audio frame is the time-domain encoding mode, the encoding mode of the previous audio frame is the frequency-domain encoding mode to the time-domain encoding switching encoding mode, the final encoding mode of the audio frame is determined as the time-domain encoding mode.

[0024] According to the final encoding mode of the audio frame, the following encoding methods can be used:

[0025] (1) The final encoding mode of the audio frame is the time-domain encoding mode

[0026] The audio sending device splices the last N / 2 data of the previous audio frame and the first N / 2 data of the audio frame to obtain first data with a length of N, N representing the length of the audio frame; and performs time-domain encoding on the first data with the length of N.

[0027] In the present application, the time-domain encoding process of the audio frame can refer to the LPC encoding mode shown in Figure 4 The difference is that: Figure 4 In the above, the encoding object is a complete audio frame, which includes data on, for example, N sampling points; while in the present application, the encoding object is also a data with a length of N, but the data is spliced by the last N / 2 data of the previous audio frame and the first N / 2 data of the audio frame, which is equivalent to containing half data of two audio frames respectively.

[0028] Since the time delay of the time-domain encoding mode is lower than that of the frequency-domain encoding mode, in order to facilitate the switching between the time-domain encoding mode and the frequency-domain encoding mode, after receiving the code stream of the current audio frame, the audio receiving device performs time-domain decoding to obtain the first N / 2 data of the data with the length of N, and the last N / 2 data of the data with the length of N obtained by the last time-domain decoding to form the N data of the complete previous audio frame. In this way, the total time delay of the time-domain encoding and decoding can be changed from N sampling points to 2N sampling points, so as to realize the alignment of the total time delay of the frequency-domain encoding and decoding. The last N / 2 data of the data with the length of N obtained by decoding is stored in the cache, so as to form the N data of the complete current audio frame by the same method after the decoding of the next audio frame.

[0029] (2) The final encoding mode of the audio frame is the frequency-domain encoding mode

[0030] The audio sending device splices the N data of the previous audio frame and the N data of the audio frame to obtain first data with a length of 2N, N representing the length of the audio frame; performs windowing processing on the first data with the length of 2N to obtain second data with the length of 2N; and performs frequency-domain encoding on the second data with the length of 2N.

[0031] In the present application, the frequency-domain encoding process of the audio frame can refer to the LPC encoding mode shown in Figures 5 to 7The MDCT transform mode is shown. As described above, the total time delay of the frequency domain coding is 2N samples.

[0032] (3) The final coding mode of the audio frame is switched from the frequency domain coding to the time domain coding

[0033] The audio sending device performs windowing processing on the N data of the previous audio frame to obtain first data with a length of N, N representing the length of the audio frame; performs odd symmetry folding processing with the center position as the odd symmetry point on the first data with the length of N to obtain second data with a length of N / 2; splices the second data with the length of N / 2 and the first N / 2 data of the audio frame to obtain third data with a length of N; and performs time domain coding on the third data with the length of N.

[0034] The time domain coding process of the audio frame in the present application can refer to the LPC coding mode shown in Figure 4 The difference lies in that before the time domain coding, the frequency domain coding mode (for example, the windowing processing and the odd symmetry folding processing) and the time domain coding mode are fused, the respective partial data of the previous audio frame and the audio frame are preprocessed to obtain data with a length of N, and then the data with the length of N is coded in the time domain.

[0035] (4) The final coding mode of the audio frame is switched from the time domain coding to the frequency domain coding

[0036] a. The total time delay of the time domain coding and decoding is N samples, N representing the length of the audio frame

[0037] The audio sending device performs windowing processing on the N data of the audio frame to obtain first data with a length of N; performs even symmetry folding processing with the center position as the even symmetry point on the first data with the length of N to obtain second data with a length of N / 2; splices the last N / 2 data of the previous audio frame and the second data with the length of N / 2 to obtain third data with a length of N; and performs time domain coding on the third data with the length of N.

[0038] The time domain coding process of the audio frame in the present application can refer to the LPC coding mode shown in Figure 4 The difference lies in that before the time domain coding, the frequency domain coding mode (for example, the windowing processing and the even symmetry folding processing) and the time domain coding mode are fused, the respective partial data of the previous audio frame and the audio frame are preprocessed to obtain data with a length of N, and then the data with the length of N is coded in the time domain.

[0039] b. The total time delay of the time domain coding and decoding is N+D samples, N representing the length of the audio frame, and D representing the time delay of the quadrature mirror filter (QMF) in the time domain coding mode

[0040] The audio sending device performs windowing processing on the N data of the audio frame to obtain first data with a length of N; performs even symmetry folding processing on the first data with the length of N with a center position as an even symmetry point to obtain second data with a length of N / 2; performs even symmetry unfolding processing on the second data with the length of N / 2 with the center position as the even symmetry point to obtain third data with a length of N, the first N / 2 data of the third data with the length of N is the second data with the length of N / 2, and the last N / 2 data of the third data with the length of N is fourth data with the length of N / 2; splices the first D data of the second data with the length of N / 2 and the fourth data with the length of N / 2 to obtain fifth data with a length of N / 2+D; performs interpolation processing on the fifth data with the length of N / 2+D to obtain sixth data with a length of N / 2; splices the last N / 2 data of the previous audio frame and the sixth data with the length of N / 2 to obtain seventh data with a length of N; and performs time domain encoding on the seventh data with the length of N.

[0041] The time domain encoding process of the audio frame in the present application can refer to the LPC encoding mode shown in Figure 4 The difference between the b case and the a case is that, because the time domain encoding mode increases the QMF processing to divide the frequency band of the audio to be transmitted into multiple sub-bands, the time delay of the time domain encoding mode is increased by D samples, and therefore, in order to align the total time delay of the frequency domain encoding and decoding and the time domain encoding and decoding, interpolation processing is needed in the preprocessing to interpolate the audio frame from N / 2-D data or N / 2+D data to N / 2 data at equal intervals. Because the value of D is much smaller than N / 2 in actual situations, the above interpolation processing cannot be perceived by people.

[0042] According to the above (3) and (4) description, when the final encoding mode of the audio frame is the switching encoding mode (switching from the frequency domain encoding to the time domain encoding and switching from the time domain encoding to the frequency domain encoding), the N data of the audio frame are encoded by using the time domain encoding mode, which can greatly reduce the running overhead, reduce the code stream bit size (i.e. constant code rate) under the premise of maintaining the sound quality, and has very important significance for improving the anti-interference of short distance transmission. In addition, the audio sending device constructs new to-be-encoded data through windowing processing and odd / even symmetry folding processing, and the audio receiving device recovers the original data through odd / even symmetry unfolding processing and de-windowing processing, and superposition processing after decoding the code stream. It can be seen that this method only needs to change the data before encoding and after decoding, which can be used as pre-processing before encoding and post-processing after decoding, and does not change the internal core algorithm of the time domain encoding and decoding, and has strong practicability. Moreover, the present application can be combined with any time domain encoding mode and frequency domain encoding mode, which only needs to meet the following conditions: a, the lengths of the audio frames are the same; b, the delays are alignable.

[0043] The present application can support the encoding and decoding of audio frames of speech type and music type by fusing the frequency domain encoding mode and the time domain encoding mode, and can be applied to low delay scenarios and high sound quality scenarios, realizing the full code rate range of 16 kbps-1000 kbps, having the advantages of simple implementation, small overhead, constant code rate, high sound quality, and adaptive switching of different encoding modes according to signal characteristics and encoding modes. In addition, any time domain encoding mode and any frequency domain encoding mode can be fused to make the architecture of audio encoding and decoding flexible and have strong expandability.

[0044] In a second aspect, the present application provides an audio decoding method, comprising: obtaining a code stream; parsing the code stream to obtain code stream data to be decoded and indication information of a decoding mode, the decoding mode comprising a time domain decoding mode, a frequency domain decoding mode, a time domain decoding to frequency domain decoding switching decoding mode, or a frequency domain decoding to time domain decoding switching decoding mode; and decoding the code stream data according to the decoding mode.

[0045] The audio receiving device parses the data packet header of the received encoded code stream to obtain the indication information of the decoding mode, decodes and post-processes (odd / even symmetry unfolding processing, interpolation processing, de-windowing processing, etc.) the code stream data according to the decoding mode. The audio receiving device parses the data packet header to determine the decoding mode of the current audio frame. Then the audio receiving device decodes (decodes or decodes and post-processes) the code stream data by using the decoding mode to obtain audio PCM data.

[0046] Corresponding to the encoding process, there are the following decoding methods according to the decoding mode:

[0047] (1) The decoding mode of the audio frame is the time domain decoding mode

[0048] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of N, the first N / 2 data of the first data with the length of N are the last N / 2 data of the last audio frame, the last N / 2 data of the first data with the length of N are the first N / 2 data of the current audio frame, and N represents the length of the audio frame; the N / 2 data in the cache obtained by the last time domain decoding and the first N / 2 data of the first data with the length of N are spliced to obtain the last audio frame; and the last N / 2 data of the first data with the length of N is stored in the cache.

[0049] (2) the decoding mode of the audio frame is a frequency domain decoding mode, the total time delay of the time domain coding and decoding is N+D sample points, N represents the length of the audio frame, D represents the time delay of the QMF in the time domain coding mode, and the decoding mode of the last audio frame is a time domain decoding mode switching to a frequency domain decoding mode

[0050] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of 2N; the first N / 2 data of the first N data of the first data with the length of 2N and the following D data are spliced to obtain second data with a length of N / 2+D; the second data with the length of N / 2+D is interpolated to obtain third data with a length of N / 2; the last N / 2 data of the first N data of the first data with the length of 2N and the previous D data are spliced to obtain fourth data with a length of N / 2+D; the fourth data with the length of N / 2+D is interpolated to obtain fifth data with a length of N / 2; the third data with the length of N / 2 and the fifth data with the length of N / 2 are spliced to obtain sixth data with a length of N; the sixth data with the length of N and the N data in the cache obtained by the last time domain decoding are windowed and overlapped to obtain the last audio frame; and the last N data of the first data with the length of 2N is stored in the cache.

[0051] (3) the decoding mode of the audio frame is a frequency domain decoding mode, the total time delay of the time domain coding and decoding is N+D sample points, N represents the length of the audio frame, D represents the time delay of the QMF in the time domain coding mode, and the decoding mode of the last audio frame is a frequency domain decoding mode; or the decoding mode of the audio frame is a frequency domain decoding mode, and the total time delay of the time domain coding and decoding is N sample points

[0052] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of 2N; the first N data of the first data with the length of 2N and the last N data of the second data with the length of 2N obtained by the last time domain decoding are windowed and overlapped to obtain the last audio frame; and the last N data of the first data with the length of 2N is stored in the cache.

[0053] (4) the decoding mode of the audio frame is a frequency domain decoding mode switching to a time domain decoding mode, and the total time delay of the time domain coding and decoding is N sample points, N represents the length of the audio frame

[0054] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of N; performs odd symmetry expansion processing on the first N / 2 data of the first data with the length of N with a center position as an odd symmetry point to obtain second data with a length of N; performs window-removing overlap processing on the second data with the length of N and the last N data of third data with a length of 2N obtained by the last frequency domain decoding to obtain a last audio frame with a length of N; and stores the first N / 2 data of the first data with the length of N in the buffer.

[0055] (5) The decoding mode of the audio frame is switched from frequency domain decoding to time domain decoding, and the total time delay of the time domain coding and decoding is N+D sample points, N represents the length of the audio frame, and D represents the time delay of the QMF in the time domain coding mode

[0056] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of N; performs interpolation processing on the last N / 2-D data of the first N / 2 data of the first data with the length of N to obtain second data with a length of N / 2; performs odd symmetry expansion processing on the second data with the length of N / 2 with a center position as an odd symmetry point to obtain third data with a length of N; performs interpolation processing on the first N / 2-D data of the last N data of fourth data with a length of 2N obtained by the last frequency domain decoding to obtain fifth data with a length of N / 2; performs interpolation processing on the last N / 2-D data of the last N data of the fourth data with the length of 2N obtained by the last frequency domain decoding to obtain sixth data with a length of N / 2; splices the fifth data with the length of N / 2 and the sixth data with the length of N / 2 to obtain seventh data with a length of N; performs window-removing overlap processing on the third data with the length of N and the seventh data with the length of N to obtain a last audio frame with a length of N; and stores the last N / 2 data of the first data with the length of N in the buffer.

[0057] (6) The decoding mode of the audio frame is switched from time domain decoding to frequency domain decoding

[0058] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of N, N representing the length of the audio frame; splices the last N / 2 data of second data with the length of N obtained by the last time domain decoding and the first N / 2 data of the first data with the length of N to obtain a last audio frame with a length of N; performs even symmetry expansion processing on the last N / 2 data of the first data with the length of N with a center position as an even symmetry point to obtain third data with a length of N; and stores the third data with the length of N in the buffer.

[0059] The application can support coding and decoding of audio frames of voice type and music type by fusing a frequency domain coding mode and a time domain coding mode, and can be applied to low delay scenarios and high sound quality scenarios, implement a full code rate range of 16 kbps-1000 kbps, and has the advantages of simple implementation, small overhead, constant code rate, high sound quality, and adaptive switching of different coding modes according to signal characteristics and coding modes. In addition, any time domain coding mode and any frequency domain coding mode can be fused to make the architecture of audio coding and decoding flexible and strong in expandability.

[0060] In a possible implementation, the object audio frame coded on the audio sending device can be a PCM signal, or can be a signal obtained by performing QMF on the PCM signal, or can be a residual signal obtained by performing inverse LPC filtering on the PCM signal.

[0061] The application can directly perform the audio coding and decoding method on the original PCM signal. Alternatively, the audio coding and decoding can also be performed on the LPC filtered residual signal and the QMF split subband signal respectively. One is that the audio sending device obtains a residual signal by performing inverse LPC filtering on the original PCM signal, and then performs the audio coding method on the residual signal. The audio receiving device recovers the residual signal according to the audio decoding method, and then performs LPC filtering on the residual signal to synthesize the original PCM signal. The other is that the audio sending device splits the original PCM signal into multiple subband signals by QMF, and then performs the audio coding method on each subband signal. The audio receiving device recovers the multiple subband signals according to the audio decoding method, and then performs QMF on the multiple subband signals to synthesize the original PCM signal.

[0062] In a third aspect, the application provides an audio coding apparatus, comprising: a determination module configured to determine a signal type and a use scenario of a current audio frame to be coded, wherein the use scenario comprises a low delay scenario or a high sound quality scenario; determine an initial coding mode of the current audio frame according to the signal type of the current audio frame and the use scenario of the current audio frame, wherein the initial coding mode comprises a time domain coding mode or a frequency domain coding mode; determine a final coding mode of the current audio frame according to the initial coding mode of the current audio frame and a final coding mode of a previous audio frame, wherein the final coding mode comprises the time domain coding mode, the frequency domain coding mode, a time domain coding to frequency domain coding switching coding mode, or a frequency domain coding to time domain coding switching coding mode; and a coding module configured to code the current audio frame according to the final coding mode of the current audio frame.

[0063] In a possible implementation, the determining module is specifically configured to: when the signal type of the current audio frame is a music type and the use scenario is a low-delay scenario, determine that the initial encoding mode of the current audio frame is a time-domain encoding mode; or when the signal type of the current audio frame is a music type and the use scenario is a high-audio-quality scenario, determine whether the current audio frame is a time-domain stationary signal; if the current audio frame is a time-domain stationary signal, determine that the initial encoding mode of the current audio frame is a frequency-domain encoding mode; or if the current audio frame is not a time-domain stationary signal, determine that the initial encoding mode of the current audio frame is a time-domain encoding mode.

[0064] In a possible implementation, the determining module is specifically configured to: split the audio frame into a plurality of audio subframes; calculate energy variances of the plurality of audio subframes; when the energy variances of the plurality of audio subframes are greater than a first preset threshold, filter the audio frame by using a linear prediction coding (LPC) filter to obtain energy of a filtered audio frame, and determine whether the audio frame is the time-domain stationary signal according to the energy of the filtered audio frame; when the energy variances of the plurality of audio subframes are less than the first preset threshold, transform the audio frame by using a modified discrete cosine transform (MDCT) to obtain energy variances of each frequency point of a transformed audio frame, and determine whether the audio frame is the time-domain stationary signal according to the energy variances of each frequency point of the transformed audio frame.

[0065] In a possible implementation, the determining module is specifically configured to: if a ratio of the energy of the filtered audio frame to the energy of the audio frame is greater than a second preset threshold, the audio frame is the time-domain stationary signal; or if the ratio of the energy of the filtered audio frame to the energy of the audio frame is less than the second preset threshold, the audio frame is not the time-domain stationary signal.

[0066] In a possible implementation, the determining module is specifically configured to: if the energy variances of each frequency point of the transformed audio frame are greater than a third preset threshold, the audio frame is the time-domain stationary signal; or if the energy variances of each frequency point of the transformed audio frame are less than the third preset threshold, the audio frame is not the time-domain stationary signal.

[0067] In a possible implementation, the determining module is specifically configured to: when the initial encoding mode of the current audio frame and the final encoding mode of the previous audio frame are both frequency domain encoding modes, determine that the final encoding mode of the current audio frame is a frequency domain encoding mode; or when the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a time domain encoding mode, determine that the final encoding mode of the current audio frame is a time domain to frequency domain switching encoding mode; or when the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a time domain to frequency domain switching encoding mode, determine that the final encoding mode of the current audio frame is a frequency domain encoding mode; or when the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain to time domain switching encoding mode, determine that the final encoding mode of the current audio frame is a time domain to frequency domain switching encoding mode; or when the initial encoding mode of the current audio frame and the final encoding mode of the previous audio frame are both time domain encoding modes, determine that the final encoding mode of the current audio frame is a time domain encoding mode; or when the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain encoding mode, determine that the final encoding mode of the current audio frame is a frequency domain to time domain switching encoding mode; or when the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a time domain to frequency domain switching encoding mode, determine that the final encoding mode of the current audio frame is a frequency domain to time domain switching encoding mode; or when the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain to time domain switching encoding mode, determine that the final encoding mode of the current audio frame is a time domain encoding mode.

[0068] In a possible implementation, when the final encoding mode is the time domain encoding mode, the encoding module is specifically configured to: splice the last N / 2 sample data of the previous audio frame and the first N / 2 sample data of the current audio frame to obtain first data with a length of N, N representing the length of an audio frame; and perform time domain encoding on the first data.

[0069] In a possible implementation, when the final encoding mode is the frequency domain encoding mode, the encoding module is specifically configured to: splice the N sample data of the previous audio frame and the N sample data of the current audio frame to obtain second data with a length of 2N, N representing the length of an audio frame; perform windowing processing on the second data to obtain third data with a length of 2N; and perform frequency domain encoding on the third data.

[0070] In a possible implementation, when the final encoding mode is the frequency-domain encoding to time-domain encoding switching encoding mode, the encoding module is specifically configured to perform windowing processing on the N sample data of the previous audio frame to obtain fourth data with a length of N, N representing the length of an audio frame; perform odd symmetry folding processing on the fourth data with a center position as an odd symmetry point to obtain fifth data with a length of N / 2; splice the fifth data and the first N / 2 sample data of the current audio frame to obtain sixth data with a length of N; and perform time-domain encoding on the sixth data.

[0071] In a possible implementation, when the final encoding mode is the time-domain encoding to frequency-domain encoding switching encoding mode, and the total time delay of time-domain encoding and decoding is N sample data, the encoding module is specifically configured to perform windowing processing on the N sample data of the current audio frame to obtain seventh data with a length of N, N representing the length of an audio frame; perform even symmetry folding processing on the seventh data with a center position as an even symmetry point to obtain eighth data with a length of N / 2; splice the last N / 2 sample data of the previous audio frame and the eighth data to obtain ninth data with a length of N; and perform time-domain encoding on the ninth data.

[0072] In a possible implementation, when the final encoding mode is the time-domain encoding to frequency-domain encoding switching encoding mode, and the total time delay of time-domain encoding and decoding is N+D sample data, the encoding module is specifically configured to perform windowing processing on the N sample data of the current audio frame to obtain tenth data with a length of N; perform even symmetry folding processing on the tenth data with a center position as an even symmetry point to obtain eleventh data with a length of N / 2, N representing the length of an audio frame; perform even symmetry unfolding processing on the eleventh data with a center position as an even symmetry point to obtain twelfth data with a length of N, the first N / 2 sample data of the twelfth data being the eleventh data, and the last N / 2 sample data of the twelfth data being thirteenth data with a length of N / 2; splice the eleventh data and the first D sample data of the thirteenth data to obtain fourteenth data with a length of N / 2+D, D representing the time delay of QMF in the time-domain encoding mode; perform interpolation processing on the fourteenth data to obtain fifteenth data with a length of N / 2; splice the last N / 2 sample data of the previous audio frame and the fifteenth data to obtain sixteenth data with a length of N; and perform time-domain encoding on the sixteenth data.

[0073] In a possible implementation, the current audio frame is a pulse code modulation (PCM) signal; or the current audio frame is a signal obtained by performing quadrature mirror filtering (QMF) on a PCM signal; or the current audio frame is a residual signal obtained by performing inverse LPC filtering on a PCM signal.

[0074] In a fourth aspect, the present application provides an audio decoding apparatus, comprising: an obtaining module, configured to obtain a bitstream; a decoding module, configured to parse the bitstream to obtain bitstream data to be decoded and indication information of a decoding mode, the decoding mode comprising a time-domain decoding mode, a frequency-domain decoding mode, a time-domain decoding to frequency-domain decoding switching decoding mode, or a frequency-domain decoding to time-domain decoding switching decoding mode; and decode the bitstream data according to the decoding mode.

[0075] In a possible implementation, when the decoding mode is the time-domain decoding mode, the decoding module is specifically configured to perform time-domain decoding on the bitstream data to obtain a seventeenth data with a length of N, the last N / 2 sample data of the seventeenth data being the first N / 2 sample data of a current audio frame, the first N / 2 sample data of the seventeenth data being the last N / 2 sample data of a previous audio frame of the current audio frame, N representing the length of an audio frame; splice the first N / 2 sample data of the previous audio frame and the first N / 2 sample data of the seventeenth data to obtain the data of the previous audio frame; and store the last N / 2 sample data of the seventeenth data in a buffer.

[0076] In a possible implementation, when the decoding mode is the frequency-domain decoding mode, the total time delay of time-domain coding and decoding is N+D sample data, and the decoding mode of a previous audio frame is the time-domain decoding to frequency-domain decoding switching decoding mode, the decoding module is specifically configured to perform frequency-domain decoding on the bitstream data to obtain an eighteenth data with a length of 2N, N representing the length of an audio frame; splice the first N / 2 sample data of the first N sample data of the eighteenth data and D sample data after the first N / 2 sample data to obtain a nineteenth data with a length of N / 2+D, D representing the time delay of a Quadrature Mirror Filter (QMF) in the time-domain coding mode; perform interpolation processing on the nineteenth data to obtain a twentieth data with a length of N / 2; splice the last N / 2 sample data of the first N sample data of the eighteenth data and D sample data before the last N / 2 sample data to obtain a twenty-first data with a length of N / 2+D; perform interpolation processing on the twenty-first data to obtain a twenty-second data with a length of N / 2; splice the twentieth data and the twenty-second data to obtain a twenty-third data with a length of N; perform windowing overlap processing on the twenty-third data and N sample data in the buffer obtained by previous frequency-domain decoding to obtain the data of the previous audio frame; and store the last N sample data of the eighteenth data in the buffer.

[0077] In a possible implementation, when the decoding mode is the frequency domain decoding mode, and the total time delay of time domain coding and decoding is N+D sample data, N represents the length of an audio frame, D represents the time delay of QMF in the time domain coding mode, and the decoding mode of a previous audio frame is the frequency domain decoding mode; or when the decoding mode is the frequency domain decoding mode, and the total time delay of time domain coding and decoding is N sample data, the decoding module is specifically configured to perform frequency domain decoding on the code stream data to obtain twenty-fourth data with a length of 2N; perform windowing overlap processing on the first N sample data of the twenty-fourth data and the last N sample data of twenty-fifth data with a length of 2N obtained by previous frequency domain decoding to obtain data of a previous audio frame; and store the last N sample data of the twenty-fourth data in a buffer.

[0078] In a possible implementation, when the decoding mode is the frequency domain decoding mode, and the total time delay of time domain coding and decoding is N+D sample data, N represents the length of an audio frame, D represents the time delay of QMF in the time domain coding mode, and the decoding mode of a previous audio frame is the frequency domain decoding mode; or when the decoding mode is the frequency domain decoding mode, and the total time delay of time domain coding and decoding is N sample data, the decoding module is specifically configured to perform frequency domain decoding on the code stream data to obtain twenty-fourth data with a length of 2N; perform windowing overlap processing on the first N sample data of the twenty-fourth data and the last N sample data of twenty-fifth data with a length of 2N obtained by previous frequency domain decoding to obtain data of a previous audio frame; and store the last N sample data of the twenty-fourth data in a buffer.

[0079] In a possible implementation, when the decoding mode is switched from the frequency domain decoding to the time domain decoding, and the total time delay of the time domain coding is N+D sample data, N represents the length of an audio frame, and D represents the time delay of the QMF in the time domain coding mode, the decoding module is specifically configured to perform time domain decoding on the code stream data to obtain a twenty-ninth data with a length of N; perform interpolation processing on the last N / 2-D sample data of the first N / 2 sample data of the twenty-ninth data to obtain a thirtieth data with a length of N / 2; perform odd symmetric expansion processing on the thirtieth data with a center position as an odd symmetric point to obtain a thirty-first data with a length of N; perform interpolation processing on the first N / 2-D sample data of the last N sample data of a thirty-second data with a length of 2N obtained by last frequency domain decoding to obtain a thirty-third data with a length of N / 2; perform interpolation processing on the last N / 2-D sample data of the last N sample data of the thirty-second data to obtain a thirty-fourth data with a length of N / 2; splice the thirty-third data and the thirty-fourth data to obtain a thirty-fifth data with a length of N; perform windowing overlap processing on the thirty-first data and the thirty-fifth data to obtain data of a last audio frame with a length of N; and store the last N / 2 sample data of the twenty-ninth data in a buffer.

[0080] In a possible implementation, when the decoding mode is switched from the time domain decoding to the frequency domain decoding, the decoding module is specifically configured to perform time domain decoding on the code stream data to obtain a thirty-sixth data with a length of N, N representing the length of an audio frame; splice the last N / 2 sample data of a thirty-seventh data with a length of N obtained by last time domain decoding and the first N / 2 sample data of the thirty-sixth data to obtain data of a last audio frame with a length of N; perform even symmetric expansion processing on the last N / 2 sample data of the thirty-sixth data with a center position as an even symmetric point to obtain a thirty-eighth data with a length of N; and store the thirty-eighth data in a buffer.

[0081] In a possible implementation, the last audio frame is a pulse code modulation (PCM) signal; or, the last audio frame is a PCM signal obtained by a quadrature mirror filter (QMF); or, the last audio frame is a PCM signal obtained by an LPC filter.

[0082] In a fifth aspect, the present application provides an audio encoding device, including: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method in any of the above first aspect.

[0083] In a sixth aspect, the present application provides an audio decoding device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of the above second aspects.

[0084] In a seventh aspect, the present application provides a computer readable storage medium, comprising a computer program, when the computer program is executed on a computer, the computer program causes the computer to execute the method of any one of the above first to second aspects.

[0085] In an eighth aspect, the present application provides a computer readable storage medium, comprising an encoded bitstream obtained according to the audio encoding method of any one of the above first aspects.

[0086] In a ninth aspect, the present application provides a computer readable storage medium, comprising an audio frame obtained according to the audio decoding method of any one of the above second aspects. BRIEF DESCRIPTION OF DRAWINGS

[0087] Figure 1 An exemplary structural diagram of an audio playing system of the present application;

[0088] Figure 2 An exemplary structural block diagram of an audio decoding system 10 of the present application;

[0089] Figure 3 An exemplary flow chart of an audio encoding method of the present application;

[0090] Figure 4 An exemplary coding and decoding block diagram of an LPC encoding mode of the present application;

[0091] Figure 5 An exemplary coding and decoding block diagram of an MDCT encoding mode of the present application;

[0092] Figures 6a to 6c Several exemplary diagrams of window functions of the present application;

[0093] Figure 7 An exemplary process diagram of obtaining PCM data through MDCT transform and IMDCT transform of the present application;

[0094] Figure 8 An exemplary flow chart of a determination method of an encoding mode of an audio frame of the present application;

[0095] Figure 9 An exemplary flow chart of switching coding and decoding modes of the present application;

[0096] Figure 10An exemplary frame change flow chart for switching encoding mode from time domain encoding to frequency domain encoding of the present application;

[0097] Figure 11 An exemplary frame change flow chart for switching encoding mode from time domain encoding to frequency domain encoding of the present application;

[0098] Figure 12a And 12b An exemplary flow chart for switching encoding / decoding mode of the present application;

[0099] Figure 13 An exemplary frame change flow chart for switching encoding mode from time domain encoding to frequency domain encoding of the present application;

[0100] Figure 14 An exemplary frame change flow chart for switching encoding mode from time domain encoding to frequency domain encoding of the present application;

[0101] Figure 15 An exemplary structure diagram of audio encoding device embodiment of the present application;

[0102] Figure 16 An exemplary structure diagram of audio decoding device embodiment of the present application. DETAILED DESCRIPTION

[0103] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0104] The terms "first", "second", etc. in the description, claims and drawings of the present application are only used for distinguishing the purposes of description, and cannot be understood as indicating or implying relative importance, nor can be understood as indicating or implying sequence. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, inclusion of a series of steps or units. The method, system, product or device does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0105] It should be understood that in the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0106] Related terms explained in the present application:

[0107] Audio frame: Audio data is streaming. In actual application, in order to facilitate audio processing and transmission, the amount of audio data in a time length is usually taken as an audio frame, and this time length is called "sampling time". Its value can be determined according to the needs of the codec and specific application, for example, the time length is 2.0ms-60ms, ms is millisecond.

[0108] Figure 1 An exemplary structure diagram of the audio playback system of the present application is shown in Figure 1 The audio playback system includes an audio sending device and an audio receiving device, wherein the audio sending device includes, for example, a mobile phone, a computer (notebook computer, desktop computer, etc.), a tablet (handheld tablet, vehicle-mounted tablet, etc.) and other devices that can encode and send audio code streams; The audio receiving device includes, for example, TWS earphones, ordinary wireless earphones, sound equipment, smart watches, smart glasses and other devices that can receive audio code streams, decode audio code streams and play.

[0109] A Bluetooth connection can be established between the audio sending device and the audio receiving device, and voice and music transmission can be supported between the two. A more widely used example of the audio sending device and the audio receiving device is between a mobile phone and TWS earphones, wireless headphones or wireless neck-type earphones, or between a mobile phone and other terminal devices (such as smart speakers, smart watches, smart glasses, and vehicle-mounted sound boxes). Alternatively, an example of the audio sending device and the audio receiving device can also be between a tablet, a notebook computer or a desktop computer and TWS earphones, wireless headphones, wireless neck-type earphones or other terminal devices (such as smart speakers, smart watches, smart glasses, and vehicle-mounted sound boxes).

[0110] It should be noted that, in addition to the Bluetooth connection, the audio sending device and the audio receiving device can also be connected through other communication modes, such as a WiFi connection, a wired connection, or other wireless connections, and the present application does not make a specific limitation thereto.

[0111] Figure 2 An exemplary structural block diagram of the audio decoding system 10 of the present application is shown in FIG. 1, which can include a source device 12 and a destination device 14. The source device 12 can be an audio sending device, and the destination device 14 can be an audio receiving device. Figure 2 Figure 1 The source device 12 generates an encoded bitstream, and thus, the source device 12 can be referred to as an audio encoding apparatus. The destination device 14 can decode the encoded bitstream generated by the source device 12, and thus, the destination device 14 can be referred to as an audio decoding apparatus. Figure 1

[0112] The source device 12 includes an encoder 20, and optionally, an input interface 16, an audio pre-processor 18, and a communication interface 22.

[0113] The input interface 16 is configured to input audio pulse code modulation (PCM) data and a usage scenario. The audio PCM data can be classified as a speech type or a music type, and the usage scenario can be classified as a low-delay scenario or a high-quality scenario.

[0114] The audio pre-processor 18 is configured to determine an encoding mode according to the type of the audio PCM data and the usage scenario input by the input interface 16. That is, in the speech type, or in the music type and the low-delay scenario, a time-domain encoding mode (for example, an LPC mode) is adopted to encode an audio frame, so that the advantages of the time-domain encoding mode in efficient encoding of a speech signal and low delay can be fully reflected. In the music type and the high-quality scenario, an encoding scheme that combines a time-domain encoding mode and a frequency-domain encoding mode (for example, an MDCT mode) is adopted to encode the audio frame, and a suitable encoding mode is adaptively selected according to the transient characteristics and the stationary characteristics of the audio frame.

[0115] The encoder 20 is configured to encode the audio frame according to the encoding mode determined by the audio pre-processor 18 to obtain a bitstream.

[0116] The communication interface 22 in the source device 12 can be configured to receive the bitstream and transmit the bitstream to the destination device 14 through a communication channel 13.

[0117] The destination device 14 includes a decoder 30, and optionally, a communication interface 28, an audio post-processor 32, and a playback device 34.

[0118] ​​The communication interface 28 in the destination device 14 is used to directly receive the code stream from the source device 12 and provide the code stream to the decoder 30 .

[0119] The communication interface 22 and the communication interface 28 may be used to send or receive code streams via a communication link, such as a Bluetooth connection, between the source device 12 and the destination device 14 .

[0120] For example, the communication interface 22 may be used to encapsulate the code stream into a suitable format such as a message, and / or process the code stream using Bluetooth transmission coding or processing for transmission over the communication link.

[0121] The communication interface 28 corresponds to the communication interface 22, and can be used, for example, to receive a code stream and use corresponding transmission decoding or processing and / or decapsulation to obtain the code stream.

[0122] Both the communication interface 22 and the communication interface 28 can be configured as follows Figure 2 The unidirectional communication interface or the bidirectional communication interface indicated by the arrow pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14 can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded audio data, etc.

[0123] The decoder 30 is used to receive a code stream and decode the code stream data in the code stream according to the indication information of the decoding mode in the code stream to obtain audio data.

[0124] The audio post-processor 32 is configured to perform post-processing on the decoded audio data to obtain post-processed audio data. The post-processing performed by the audio post-processor 32 may include, for example, trimming or resampling.

[0125] The playback device 34 is used to receive the post-processed audio data and play the audio to the user or listener. The playback device 34 can be or include any type of player for playing the reconstructed audio, such as an integrated or external speaker. For example, the speaker can include a speaker, a stereo, etc.

[0126] Based on the description of the above embodiments, the present application provides a method for encoding and decoding a multi-channel audio signal.

[0127] Figure 3 This is an exemplary flow chart of the audio encoding method of the present application. The process 300 can be performed by an audio transmitting device in an audio playback system, that is, the audio transmitting device performs audio encoding and then sends the encoded code stream to the audio receiving device. The process 300 is described as a series of steps or operations. It should be understood that the process 300 can be performed in various orders and / or occur simultaneously, not limited to Figure 3 The execution order shown in Figure 3 As shown, the method includes:

[0128] In step 301, the audio sending device determines the signal type and use scenario of a current audio frame to be encoded.

[0129] The audio frame can be any frame in the audio transmitted by the audio sending device to the audio receiving device. In this application, the object to be encoded each time can be a frame of audio in the audio, that is, the audio encoding method provided in this application is for a frame of audio, and the method of determining the encoding mode below is applicable to each frame of audio. Therefore, in order to distinguish, the audio frame being encoded by the audio sending device is referred to as the audio frame or the current audio frame, and the audio frame encoded by the audio sending device only before the audio frame is referred to as the previous audio frame.

[0130] The signal type of the audio frame can include a speech type or a music type. The signal type of the current audio frame can be determined by the attribute information of the audio frame, for example, after the mobile phone reads the audio to be transmitted from the audio source file, the attribute information thereof can also be read from the audio source file; or the signal type of the audio frame can be determined by the playing mode of the audio frame, for example, if the user makes a call through the mobile phone, the audio frame transmitted by the mobile phone to the Bluetooth earphone belongs to the speech type, and if the user plays music on the mobile phone by using an application (APP), the audio frame transmitted by the mobile phone to the Bluetooth speaker belongs to the music type. In addition, the audio sending device can obtain the type of the audio frame by other ways, which is not limited in this application.

[0131] The use scenario of the audio frame can include a low-delay scenario or a high-audio-quality scenario. When the frame length of the codec is N, the delay of the time-domain encoding is N sample data or N+D sample data, and the delay of the frequency-domain encoding is 2N sample data or 2N-2M sample data. In this application, the default low-delay scenario refers to the time-domain encoding because the delay of the time-domain encoding is lower than that of the frequency-domain encoding. Since the frequency-domain encoding can be combined with the psychoacoustic model, the encoding quality of the time-domain stationary signal is greater than that of the time-domain encoding mode, and for the non-time-domain stationary signal, the encoding quality of the time-domain encoding mode is high. In this application, the default high-audio-quality mode refers to the encoding mode of the fusion of the time domain and the frequency domain, which has high-quality encoding effect for both the time-domain stationary signal and the non-time-domain stationary signal.

[0132] The use scenario of the audio frame can be determined by a control, an option, etc. clicked by a user on an interactive interface of an APP on the mobile phone, for example, when the user connects Bluetooth on the mobile phone, selects a low-latency scenario or a high-audio-quality scenario on a pop-up interface; or can be determined by a type of APP opened by the user on the mobile phone, for example, if the APP is a game APP, a live broadcast APP, a phone call APP or a KTV APP, etc., the use scenario is a low-latency scenario; or if the APP is an audio player APP, a video player APP, a KTV APP, a recording APP or a broadcast APP, the use scenario is a high-audio-quality scenario. In addition, the audio sending device can obtain the use scenario of the audio frame in other manners, which is not limited in the present application.

[0133] The audio frame can be any frame in audio sent by the audio sending device to the audio receiving device. In the present application, the object of each encoding can be a frame of audio in the audio, that is, the audio encoding method provided in the present application is for a frame of audio, and the method of determining the encoding mode below is applicable to each frame of audio. Therefore, in order to distinguish, the audio frame being encoded by the audio sending device is referred to as an audio frame or a current audio frame, and the audio frame being encoded by the audio sending device only before the audio frame is referred to as a previous audio frame.

[0134] Optionally, the audio frame is a pulse code modulation (PCM) signal; or, the audio frame is a signal obtained by performing a quadrature mirror filter (QMF) on the PCM signal; or, the audio frame is a residual signal obtained by performing inverse LPC filtering on the PCM signal.

[0135] Step 302, the audio sending device determines an initial encoding mode of the current audio frame according to the signal type of the current audio frame and the use scenario of the current audio frame.

[0136] The initial encoding mode includes a time domain encoding mode or a frequency domain encoding mode.

[0137] The time domain encoding mode can be, for example, an LPC encoding mode. Figure 4 For an exemplary codec framework of the LPC encoding mode of the present application, as shown in Figure 4 The audio sending device first inputs the audio PCM data into a tilt filter for processing, and the tilt filter is used to make the energy distribution of the input PCM data more uniform at different frequencies, so as to improve the efficiency of the subsequent LPC encoding. Then, the data after the tilt filtering is processed is subjected to LPC encoding. Assuming that N represents the length of the audio frame, that is, an audio frame contains N sampling points, the principle of the LPC encoding is described below:

[0138]

[0139] wherein x represents an original signal, represents the LPC coded signal, a i represents the LPC coding coefficient, p represents the order of the LPC coding, and the common order of the LPC coding is 8 or 16. The above formula represents the current sample value which can be linearly predicted by the historical p sample values x(n-i), i=l~p. i which can be obtained by solving the residual signal The minimization results in, and the common solution is the levinson durbin algorithm.

[0140] The audio transmitting device first performs the LPC analysis by the levinson durbin algorithm to obtain the p LPC coding coefficients a i , and encodes a i . Then the residual signal e(n) is obtained by the inverse LPC filtering. The residual signal e(n) is encoded by allocating fewer bits. In this way, the audio transmitting device only encodes the LPC coding coefficients and the residual signal e(n), and does not need to encode the original PCM data, so as to achieve the purpose of data compression. Finally, the audio transmitting device packs the encoded LPC coding coefficients and the residual signal e(n) into a code stream and sends the code stream to the audio receiving device for decoding.

[0141] The audio receiving device first decodes the LPC coding coefficients and the residual signal e(n), and then synthesizes the signal by the LPC filtering, and recovers the decoded PCM data by the inverse aliasing filter.

[0142] Therefore, the total delay of the time domain coding and decoding is N.

[0143] The frequency domain coding mode may be, for example, an MDCT coding mode. Figure 5 An exemplary coding and decoding framework diagram of the MDCT coding mode of the present application is shown in Figure 5 The audio transmitting device obtains the frequency domain data by performing the windowing processing and the MDCT transformation on the input audio PCM data, and then encodes the frequency domain data to obtain the encoded code stream. The audio receiving device decodes the encoded code stream, and then obtains the decoded PCM data by performing the IMDCT transformation and the de-windowing processing and aliasing addition. The windowing function is to prevent the spectrum leakage.

[0144] As shown in Figure 6a , the common window is a symmetric window, so the common window used in the MDCT transformation and the IMDCT transformation is the same. When the common window is used, the total delay of the frequency domain coding and decoding is 2N sample points, in which the coding side delay is N, and the decoding side delay is also N.

[0145] As shown in Figure 6b and Figure 6cAs shown, the low-delay window is an asymmetric window, and therefore the low-delay window for MDCT transform and the low-delay window for IMDCT transform are inverse to each other. The last M samples of the low-delay window for MDCT transform are zeros, and multiplying any data by the zeros results in zeros, so that only 2N-M data are needed for MDCT transform, which is equivalent to that the low-delay window on the encoding side has a delay of M samples less than the ordinary window in MDCT transform. The first M samples of the low-delay window for IMDCT transform are zeros, and multiplying any data by the zeros results in zeros, so that the first M samples after IMDCT transform do not need to participate in the windowing and overlapping with the previous frame, which is equivalent to that the low-delay window on the decoding side has a delay of M samples less than the ordinary window in IMDCT transform. In summary, when the low-delay window is used, the total delay of the frequency-domain encoding and decoding is 2N-2M samples.

[0146] Suppose N represents the length of an audio frame, i.e., an audio frame contains N samples, and the following exemplary describes the principle of MDCT transform and IMDCT transform:

[0147] The principle of MDCT transform is

[0148]

[0149] Correspondingly, IMDCT transform is the inverse transform of MDCT transform, and the principle is

[0150]

[0151] wherein x(n) and X(k) represent the original time-domain data and the frequency data after MDCT transform, respectively. The formula of MDCT transform above represents that the time-domain data with a length of 2N is transformed into frequency-domain data with a length of N; and the formula of IMDCT transform above represents that the frequency-domain data with a length of N is inversely transformed into time-domain data with a length of 2N.

[0152] After the audio sending device is processed by windowing and MDCT transform, frequency-domain MDCT data X(k) is obtained, and then the spectral envelope information in the frequency domain is calculated for normalizing X(k), and the normalized MDCT data is quantized by allocating bits, and is packaged into a bitstream together with the spectral envelope information. The audio receiving device decodes the spectral envelope information and the normalized MDCT data, calculates the original MDCT data, and then performs IMDCT inverse transform and windowing, and overlaps with the decoded data of the previous frame to obtain the data of the previous frame. Therefore, the frequency-domain encoder based on MDCT encodes the i-th frame, and decodes the (i-1)-th frame, and the overall delay is two frames of data.

[0153] Figure 7 An exemplary process diagram of obtaining PCM data by MDCT transform and IMDCT transform is shown in FIG. 1.Figure 7 As shown, the audio contains 3 audio frames (1st-3rd frames), each audio frame has a length of N, and is divided into two parts of N / 2 data in front and N / 2 data in back, wherein the 1st frame is divided into a and b, the 2nd frame is divided into c and d, and the 3rd frame is divided into e and f, thus a-f all have a length of N / 2.

[0154] The process of the first MDCT transformation includes that the audio sending device first splices the 1st frame and the 2nd frame into first data (a, b, c and d) with a length of 2N, then folds the first data to obtain second data (-cr-d and a-br) with a length of N, r represents data reverse, and then performs a fourth type discrete cosine transform (DCT) transformation on the second data to obtain third data (-Cr-D and A-Br) with a length of N. It can be seen that the MDCT transformation realizes the transformation from a length of 2N to a length of N.

[0155] The process of the first IMDCT transformation includes that the audio receiving device first performs a fourth type DCT inverse transformation on the third data to obtain second data (-cr-d and a-br) with a length of N, and then unfolds the second data to obtain fourth data (a-br, -ar+b, c+dr and cr+d) with a length of 2N, wherein a-br and -ar+b are mutually odd symmetric, and c+dr and cr+d are mutually even symmetric. It can be seen that the results of the first MDCT transformation and the IMDCT transformation are IMDCT(MDCT(a,b,c,d))=a-br,-ar+b,c+dr,cr+d.

[0156] Similarly, the second MDCT transformation is to splice the 2nd frame and the 3rd frame into first data (c, d, e and f) with a length of 2N, and after the above MDCT transformation and IMDCT transformation process, IMDCT(MDCT(c,d,e,f))=c-dr,-cr+d,e+fr,fr+e can be obtained.

[0157] The audio receiving device superimposes the latter half part (i.e. the latter N data c+dr and cr+d of the fourth data of the first time) obtained after the first MDCT transformation and IMDCT transformation and the former half part (i.e. the former N data c-dr and -cr+d of the fourth data of the second time) obtained after the second MDCT transformation and IMDCT transformation to obtain c and d, that is, the original 2nd frame is obtained. It can be seen that the audio receiving device can obtain the 2nd frame when the 3rd frame is input, and the overall delay is 2N samples.

[0158] In a possible implementation, for the MDCT transform and the IMDCT transform based on the low-delay window, the delay on the encoding side and the decoding side is reduced by M due to the M zero points in the window function, and the overall delay is 2N-2M.

[0159] Figure 8 An exemplary flowchart of the method for determining the encoding mode of the audio frame is shown in FIG. 1. As shown in FIG. 1, the input is PCM data of the music type, and the use scenario of the audio frame is determined. When the use scenario of the audio frame is the low-delay scenario, the encoding mode of the audio frame is determined to be the time-domain encoding mode. Figure 8

[0160] When the use scenario of the audio frame is the high-audio-quality scenario, the audio frame is split into a plurality of audio subframes, and the energy variance of the plurality of audio subframes is calculated. It is determined whether the energy variance of the plurality of audio subframes is greater than a first preset threshold.

[0161] When the energy variance of the plurality of audio subframes is greater than the first preset threshold, the audio frame is filtered by using the LPC encoding. It is determined whether the ratio of the energy of the filtered audio frame to the energy of the audio frame is greater than a second preset threshold. When the ratio of the energy of the filtered audio frame to the energy of the audio frame is greater than the second preset threshold, the audio frame is determined to be a time-domain stationary signal. When the ratio of the energy of the filtered audio frame to the energy of the audio frame is less than the second preset threshold, the audio frame is determined to be not a time-domain stationary signal.

[0162] When the energy variance of the plurality of audio subframes is less than the first preset threshold, the audio frame is transformed by using the MDCT transform. It is determined whether the energy variance of each frequency point of the transformed audio frame is greater than a third preset threshold. When the energy variance of each frequency point of the transformed audio frame is greater than the third preset threshold, the audio frame is determined to be a time-domain stationary signal. When the energy variance of each frequency point of the transformed audio frame is less than the third preset threshold, the audio frame is determined to be not a time-domain stationary signal.

[0163] When the audio frame is a time-domain stationary signal, the encoding mode of the audio frame is determined to be the frequency-domain encoding mode. When the audio frame is not a time-domain stationary signal, the encoding mode of the audio frame is determined to be the time-domain encoding mode.

[0164] It should be noted that, in the above comparisons with the first preset threshold, the second preset threshold, and the third preset threshold, the case of equality can be processed as the case of greater than or as the case of less than, which is not limited in the present application.

[0165] Optionally, when the audio frame is of the speech type, the encoding mode of the audio frame is determined to be the time-domain encoding mode. This has been adopted in the related art, and thus is not described herein.

[0166] ​It can be seen that the time-domain coding mode in the present application can be used not only for audio frames of the speech type, but also for audio frames of the music type in a low-delay scenario, and for audio frames of the music type but not time-domain stationary signals in a high-quality scenario, so that the low-delay advantage of the time-domain coding mode, especially the LPC coding mode, and the high-efficiency coding advantage for audio frames of the speech type and non-stationary signals can be fully exerted.

[0167] Step 303: The audio sending device determines the final coding mode of the current audio frame according to the initial coding mode of the current audio frame and the final coding mode of the previous audio frame.

[0168] The final coding mode includes a time-domain coding mode, a frequency-domain coding mode, a time-domain coding-to-frequency-domain coding switching coding mode, or a frequency-domain coding-to-time-domain coding switching coding mode. Step 302 determines the coding mode of the audio frame, and the final coding mode of the previous audio frame has been determined to be one of the above four final coding modes when it is coded, so there are the following cases:

[0169] (1) When the coding mode of the audio frame and the coding mode of the previous audio frame are both frequency-domain coding modes, the final coding mode of the audio frame is determined to be a frequency-domain coding mode; or,

[0170] (2) When the coding mode of the audio frame is a frequency-domain coding mode and the coding mode of the previous audio frame is a time-domain coding mode, the final coding mode of the audio frame is determined to be a time-domain coding-to-frequency-domain coding switching coding mode; or,

[0171] (3) When the coding mode of the audio frame is a frequency-domain coding mode and the coding mode of the previous audio frame is a time-domain coding-to-frequency-domain coding switching coding mode, the final coding mode of the audio frame is determined to be a frequency-domain coding mode; or,

[0172] (4) When the coding mode of the audio frame is a frequency-domain coding mode and the coding mode of the previous audio frame is a frequency-domain coding-to-time-domain coding switching coding mode, the final coding mode of the audio frame is determined to be a time-domain coding-to-frequency-domain coding switching coding mode; or,

[0173] (5) When the coding mode of the audio frame and the coding mode of the previous audio frame are both time-domain coding modes, the final coding mode of the audio frame is determined to be a time-domain coding mode; or,

[0174] (6) When the coding mode of the audio frame is a time-domain coding mode and the coding mode of the previous audio frame is a frequency-domain coding mode, the final coding mode of the audio frame is determined to be a frequency-domain coding-to-time-domain coding switching coding mode; or,

[0175] (7) When the encoding mode of the audio frame is the time-domain encoding mode, and the encoding mode of the previous audio frame is the time-domain encoding mode to the frequency-domain encoding mode switching encoding mode, the final encoding mode of the audio frame is determined to be the frequency-domain encoding mode to the time-domain encoding mode switching encoding mode; or

[0176] (8) When the encoding mode of the audio frame is the time-domain encoding mode, and the encoding mode of the previous audio frame is the frequency-domain encoding mode to the time-domain encoding mode switching encoding mode, the final encoding mode of the audio frame is determined to be the time-domain encoding mode.

[0177] Step 304, the audio sending device encodes the current audio frame according to the final encoding mode of the current audio frame.

[0178] According to the final encoding mode of the audio frame determined in step 303, there can be the following encoding methods:

[0179] (1) The final encoding mode of the audio frame is the time-domain encoding mode

[0180] The audio sending device splices the last N / 2 data of the previous audio frame and the first N / 2 data of the audio frame to obtain first data with a length of N, N representing the length of the audio frame; and performs time-domain encoding on the first data with a length of N.

[0181] The time-domain encoding process of the audio frame in the present application can refer to the LPC encoding mode shown in Figure 4 The difference is that: Figure 4 In the encoding object in the above-mentioned LPC encoding mode, it is a complete audio frame, which includes data on, for example, N sampling points; while in the present application, the encoding object is also a data with a length of N, but the data is spliced by the last N / 2 data of the previous audio frame and the first N / 2 data of the audio frame, which is equivalent to containing half data of two audio frames respectively.

[0182] Since the time-domain encoding mode has lower time delay than the frequency-domain encoding mode, in order to facilitate the switching between the time-domain encoding mode and the frequency-domain encoding mode, after receiving the code stream of the current audio frame, the audio receiving device performs time-domain decoding to obtain the first N / 2 data of the data with a length of N, and the last N / 2 data of the data with a length of N obtained by the last time-domain decoding to form the N data of the complete previous audio frame. In this way, the total time delay of the time-domain encoding and decoding can be changed from N sampling points to 2N sampling points, so as to realize the alignment of the total time delay of the frequency-domain encoding and decoding. The last N / 2 data of the data with a length of N obtained by decoding is stored in the buffer, so as to form the N data of the complete current audio frame by the above method after the decoding of the next audio frame.

[0183] (2) The final encoding mode of the audio frame is the frequency-domain encoding mode

[0184] The audio sending device splices the N data of the previous audio frame and the N data of the audio frame to obtain first data with a length of 2N, wherein N represents the length of the audio frame; performs windowing processing on the first data with the length of 2N to obtain second data with the length of 2N; and performs frequency domain coding on the second data with the length of 2N.

[0185] The frequency domain coding process of the audio frame in the present application can refer to the MDCT transformation mode shown in Figures 5 to 7 As described above, the total time delay of the frequency domain coding and decoding is 2N samples.

[0186] (3) The final coding mode of the audio frame is a switching coding mode from frequency domain coding to time domain coding

[0187] The audio sending device performs windowing processing on the N data of the previous audio frame to obtain first data with the length of N, wherein N represents the length of the audio frame; performs odd symmetry folding processing on the first data with the length of N with the center position as the odd symmetry point to obtain second data with the length of N / 2; splices the second data with the length of N / 2 and the first N / 2 data of the audio frame to obtain third data with the length of N; and performs time domain coding on the third data with the length of N.

[0188] The time domain coding process of the audio frame in the present application can refer to the LPC coding mode shown in Figure 4 The difference is that, before the time domain coding, the frequency domain coding mode (for example, the windowing processing and the odd symmetry folding processing) and the time domain coding mode are fused to pre-process the respective partial data of the previous audio frame and the audio frame to obtain data with the length of N, and then the data with the length of N is coded in the time domain.

[0189] The formula of the above-mentioned odd symmetry folding processing is:

[0190]

[0191] wherein, represents the (i+N / 2)th sample data of the audio frame, represents the (N / 2+1-i)th sample data of the audio frame, and y(i) represents the ith sample data obtained after the odd symmetry folding processing.

[0192] (4) The final coding mode of the audio frame is a switching coding mode from time domain coding to frequency domain coding

[0193] a. The total time delay of the time domain coding and decoding is N samples, wherein N represents the length of the audio frame

[0194] The audio sending device performs windowing processing on the N data of the audio frame to obtain first data with a length of N; performs even symmetry folding processing with a center position as an even symmetry point on the first data with the length of N to obtain second data with a length of N / 2; splices the last N / 2 data of the previous audio frame and the second data with the length of N / 2 to obtain third data with a length of N; and performs time domain coding on the third data with the length of N.

[0195] The time domain coding process of the audio frame in the present application can refer to the LPC coding mode shown in Figure 4 The difference lies in that, before the time domain coding, the frequency domain coding mode (for example, windowing processing and even symmetry folding processing) and the time domain coding mode are fused to pre-process the respective partial data of the previous audio frame and the audio frame to obtain data with a length of N, and then the data with the length of N is subjected to time domain coding.

[0196] The formula of the above even symmetry processing is:

[0197]

[0198] wherein, represents the (i+N / 2)th sample data of the audio frame, represents the (N / 2+1-i)th sample data of the audio frame, and y(i) represents the ith sample data obtained after the odd symmetry folding processing.

[0199] b. The total time delay of the time domain coding and decoding is N+D sample points, wherein N represents the length of the audio frame, and D represents the time delay of the quadrature mirror filter (QMF) in the time domain coding mode

[0200] The audio sending device performs windowing processing on the N data of the audio frame to obtain first data with a length of N; performs even symmetry folding processing with a center position as an even symmetry point on the first data with the length of N to obtain second data with a length of N / 2; performs even symmetry unfolding processing with the center position as the even symmetry point on the second data with the length of N / 2 to obtain third data with a length of N, wherein the first N / 2 data of the third data with the length of N is the second data with the length of N / 2, and the last N / 2 data of the third data with the length of N is fourth data with the length of N / 2; splices the first D data of the second data with the length of N / 2 and the fourth data with the length of N / 2 to obtain fifth data with a length of N / 2+D; performs interpolation processing on the fifth data with the length of N / 2+D to obtain sixth data with a length of N / 2; splices the last N / 2 data of the previous audio frame and the sixth data with the length of N / 2 to obtain seventh data with a length of N; and performs time domain coding on the seventh data with the length of N.

[0201] The time domain coding process of the audio frame in the present application can refer to the LPC coding mode shown in Figure 4The difference between the shown LPC coding mode and the a case is that, before the time domain coding, the frequency domain coding mode (such as windowing processing and even symmetry folding processing) and the time domain coding mode are fused to pre-process the respective partial data of the previous audio frame and the audio frame to obtain data with a length of N, and then the data with the length of N is coded in the time domain.

[0202] According to the descriptions of (3) and (4) above, when the final coding mode of the audio frame is the switching coding mode (switching from the frequency domain coding to the time domain coding and switching from the time domain coding to the frequency domain coding), the N data of the audio frame are coded by using the time domain coding mode, which can greatly reduce the running cost, reduce the bit size of the code stream (i.e., the code rate is constant) under the premise of maintaining the sound quality, and has very important significance for improving the anti-interference of short-distance transmission. In addition, the audio sending device constructs new data to be coded by using the windowing processing and the odd / even symmetry folding processing, and the audio receiving device recovers the original data by using the odd / even symmetry unfolding processing and the de-windowing processing and the superposition processing after decoding the code stream. It can be seen that this mode only needs to change the data before coding and after decoding, and can be used as pre-processing before coding and post-processing after decoding, without changing the internal core algorithm of the time domain coding and decoding, and has strong practicability. Moreover, the present application can be combined with any time domain coding mode and frequency domain coding mode, as long as the following conditions are met: a, the lengths of the audio frames are the same; b, the delays are alignable.

[0203] Step 305, the audio sending device sends the coded code stream to the audio receiving device.

[0204] The audio sending device can send the coded code stream to the audio receiving device by using a communication mode such as a Bluetooth connection.

[0205] Step 306, the audio receiving device parses the code stream to obtain code stream data to be decoded and indication information of the decoding mode.

[0206] The code stream includes the code stream data to be decoded and the indication information of the decoding mode, and the decoding mode includes the time domain decoding mode, the frequency domain decoding mode, the time domain decoding to frequency domain decoding switching decoding mode, or the frequency domain decoding to time domain decoding switching decoding mode.

[0207] Step 307, the audio receiving device decodes the code stream data according to the decoding mode.

[0208] The audio receiving device parses the data packet header of the received encoded code stream to obtain the indication information of the decoding mode, decodes and post-processes (odd / even symmetry unfolding processing, interpolation processing, and window removal processing, etc.) the code stream data according to the decoding mode. The audio receiving device parses the data packet header to determine the decoding mode of the current audio frame. Then the audio receiving device decodes (decodes or decodes and post-processes) the code stream data according to the decoding mode to obtain the audio PCM data.

[0209] Corresponding to the encoding process of step 304, according to the decoding mode, the following decoding methods can be used:

[0210] (1) The decoding mode of the audio frame is the time domain decoding mode

[0211] The audio receiving device performs time domain decoding on the code stream data to obtain first data with a length of N, the first N / 2 data of the first data with the length of N are the last N / 2 data of the previous audio frame, the last N / 2 data of the first data with the length of N are the first N / 2 data of the current audio frame, N represents the length of the audio frame; the N / 2 data in the cache obtained by the last time domain decoding and the first N / 2 data of the first data with the length of N are spliced to obtain the previous audio frame; the last N / 2 data of the first data with the length of N is stored in the cache.

[0212] (2) The decoding mode of the audio frame is the frequency domain decoding mode, and the total time delay of the time domain encoding and decoding is N+D samples, N represents the length of the audio frame, D represents the time delay of the QMF in the time domain encoding mode, and the decoding mode of the previous audio frame is the time domain decoding to frequency domain decoding switching decoding mode

[0213] The audio receiving device performs time domain decoding on the code stream data to obtain first data with a length of 2N; the first N / 2 data of the first N data of the first data with the length of 2N and the D data thereafter are spliced to obtain second data with a length of N / 2+D; the second data with the length of N / 2+D is interpolated to obtain third data with a length of N / 2; the last N / 2 data of the first N data of the first data with the length of 2N and the D data before are spliced to obtain fourth data with a length of N / 2+D; the fourth data with the length of N / 2+D is interpolated to obtain fifth data with a length of N / 2; the third data with the length of N / 2 and the fifth data with the length of N / 2 are spliced to obtain sixth data with a length of N; the sixth data with the length of N and the N data in the cache obtained by the last time domain decoding are windowed and overlapped to obtain the previous audio frame; the last N data of the first data with the length of 2N is stored in the cache.

[0214] (3) the decoding mode of the audio frame is a frequency domain decoding mode, and the total time delay of the time domain coding and decoding is N+D samples, N represents a length of the audio frame, D represents a time delay of a QMF in the time domain coding mode, and the decoding mode of a previous audio frame is a frequency domain decoding mode; or the decoding mode of the audio frame is a frequency domain decoding mode, and the total time delay of the time domain coding and decoding is N samples

[0215] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of 2N; performs windowing overlap processing on the first N data of the first data with the length of 2N and the last N data of second data with the length of 2N obtained by the previous frequency domain decoding to obtain the previous audio frame; and stores the last N data of the first data with the length of 2N in the buffer.

[0216] (4) the decoding mode of the audio frame is switched from a frequency domain decoding mode to a time domain decoding mode, and the total time delay of the time domain coding and decoding is N samples, N represents a length of the audio frame

[0217] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of N; performs odd symmetry expansion processing on the first N / 2 data of the first data with the length of N with a center position as an odd symmetry point to obtain second data with a length of N; performs windowing overlap processing on the second data with the length of N and the last N data of third data with a length of 2N obtained by the previous frequency domain decoding to obtain the previous audio frame with a length of N; and stores the last N / 2 data of the first data with the length of N in the buffer.

[0218] (5) the decoding mode of the audio frame is switched from a frequency domain decoding mode to a time domain decoding mode, and the total time delay of the time domain coding and decoding is N+D samples, N represents a length of the audio frame, and D represents a time delay of a QMF in the time domain coding mode

[0219] The audio receiving device performs time domain decoding on the bitstream data to obtain first data with a length of N; performs interpolation processing on the last N / 2-D data of the first N / 2 data of the first data with the length of N to obtain second data with a length of N / 2; performs odd symmetry expansion processing on the second data with the length of N / 2 with a center position as an odd symmetry point to obtain third data with a length of N; performs interpolation processing on the first N / 2-D data of the last N data of fourth data with a length of 2N obtained by the previous frequency domain decoding to obtain fifth data with a length of N / 2; performs interpolation processing on the last N / 2-D data of the last N data of the fourth data with the length of 2N obtained by the previous frequency domain decoding to obtain sixth data with a length of N / 2; splices the fifth data with the length of N / 2 and the sixth data with the length of N / 2 to obtain seventh data with a length of N; performs windowing overlap processing on the third data with the length of N and the seventh data with the length of N to obtain the previous audio frame with a length of N; and stores the last N / 2 data of the first data with the length of N in the buffer.

[0220] (6) The decoding mode of the audio frame is switched from time-domain decoding to frequency-domain decoding

[0221] The audio receiving device performs time-domain decoding on the bitstream data to obtain first data with a length of N, N representing the length of the audio frame; splices the last N / 2 data of second data with a length of N obtained by last time-domain decoding and the first N / 2 data of the first data with a length of N to obtain a last audio frame with a length of N; performs even symmetry expansion processing with a center position as an even symmetry point on the last N / 2 data of the first data with a length of N to obtain third data with a length of N; and stores the third data with a length of N in the cache.

[0222] The present application can support the coding and decoding of audio frames of speech types and music types by fusing the frequency-domain coding mode and the time-domain coding mode, can be applied to low-delay scenarios and high-audio scenarios, realizes the full-code rate range of 16 kbps-1000 kbps, has the advantages of simple implementation, small overhead, constant code rate, high audio quality, and adaptive switching of different coding modes according to signal characteristics and coding modes. In addition, any time-domain coding mode and any frequency-domain coding mode can be fused to make the architecture of audio coding and decoding flexible and strong in expandability.

[0223] In a possible implementation, the object audio frame encoded on the audio sending device can be a PCM signal, or can be a plurality of subband signals obtained by QMF splitting of the PCM signal, or can be a residual signal obtained by inverse LPC filtering of the PCM signal.

[0224] The present application can directly perform the above-mentioned audio coding and decoding method on the original PCM signal. Optionally, the audio coding and decoding can also be performed on the LPC filtered residual signal and the QMF split subband signal respectively.

[0225] One is that the audio sending device obtains a residual signal by inverse LPC filtering of the original PCM signal, and then performs the above-mentioned audio encoding method on the residual signal. The audio receiving device recovers the residual signal according to the above-mentioned audio decoding method, and then performs LPC filtering on the residual signal to synthesize the original PCM signal.

[0226] The other is that the audio sending device splits the original PCM signal into a plurality of subband signals by QMF, and then performs the above-mentioned audio encoding method on each subband signal. The audio receiving device recovers the plurality of subband signals according to the above-mentioned audio decoding method, and then performs QMF synthesis on the plurality of subband signals to synthesize the original PCM signal.

[0227] The technical solutions of the method embodiments shown in the above-mentioned figures will be described in detail in the following with several specific embodiments. Figure 3 The technical solutions of the method embodiments shown in the above-mentioned figures will be described in detail in the following with several specific embodiments.

[0228] Example 1

[0229] As described above, when the final encoding method of the audio frame is the time domain encoding method or the frequency domain encoding method, the encoding method of the audio frame in this application can refer to the description in step 304 and will not be repeated here.

[0230] Figure 9 This is an exemplary flow chart of switching the encoding and decoding mode of this application, such as Figure 9 As shown in the figure, the audio transmitter simultaneously processes the previous audio frame and the PCM data of the audio frame. After windowing and folding, it constructs data of length N. This data is then time-domain encoded and packaged into a coded bitstream. The audio receiver decodes the bitstream using time-domain decoding. It then expands, dewindows, and performs aliasing and addition on the decoded data to obtain the PCM data of the previous audio frame.

[0231] Figure 10 This is an exemplary frame change flow chart of switching the coding mode from frequency domain coding to time domain coding in this application, as shown in FIG. Figure 10 As shown in the figure, music audio in a high-quality audio scenario contains five audio frames (frames 0 to 4). Each audio frame is of length N and is equally divided into the first N / 2 data and the last N / 2 data. Frame 0 is divided into 0 and 0, frame 1 is divided into a and b, frame 2 is divided into c and d, frame 3 is divided into e and f, and frame 4 is divided into g and h. Therefore, the lengths of a to h are all N / 2. Frames 0 to 2 are time-stationary signals, while frames 3 to 4 are not.

[0232] 1. When the first frame is acquired, the final encoding method of the first frame is frequency domain encoding. The audio sending device combines the 0th frame and the 1st frame into the first data (0, 0, a and b) with a length of 2N. Figure 7 The method shown performs frequency domain encoding on the first data. The audio receiving device performs frequency domain decoding on the bitstream to obtain second data of length N. It then performs an IMDCT transform on the second data to obtain third data of length 2N (0, 0, a+br, and ar+b). Since there is no data for the -1st frame, it is assumed that the -1st frame data is 0. Therefore, the audio receiving device superimposes the first N data of the third data (0 and 0) with 0 to obtain data of frame 0, 0 and 0. The last N data of the third data (a+br and ar+b) are then stored in a buffer.

[0233] 2. When the second frame is obtained, the final encoding method of the second frame is the frequency domain encoding method. The audio sending device combines the first frame and the second frame into the first data (a, b, c and d) with a length of 2N. Figure 7The shown method performs frequency domain encoding on the first data. The audio receiving device performs frequency domain decoding on the code stream to obtain second data of length N, and then performs IMDCT transform on the second data to obtain third data of length 2N (a-br, -ar+b, c+dr and cr+d). The audio receiving device superimposes the first N data (a-br and -ar+b) of the third data with the last N data (a+br and ar+b) of the third data obtained by the last frequency domain decoding and stored in the buffer to recover the first frame (a and b). Then the last N data (c+dr and cr+d) of the third data obtained by the present frequency domain decoding is stored in the buffer.

[0234] 3. When the third frame is obtained, since the third frame is not a time domain stationary signal and the last frame is in the frequency domain encoding state, the final encoding mode of the third frame is the frequency domain encoding to time domain encoding switching encoding mode. The audio transmitting device performs folding processing on the second frame c and d to obtain first data-cr+d of length N / 2, and then forms second data (-cr+d and e) of length N by combining the first data with the first N / 2 data e of the third frame. The audio transmitting device performs time domain encoding on the second data. The audio receiving device performs time domain decoding on the code stream to obtain second data (-cr+d and e) of length N. The first N / 2 data-cr+d of the second data is first processed by odd symmetric spreading to obtain third data (c-dr and-cr+d) of length N, and then the third data (c-dr and-cr+d) is superimposed with the last N data (c+dr and cr+d) of the third data obtained by the last frequency domain decoding and stored in the buffer to recover the second frame data (c and d). Then the last N / 2 data e of the second data obtained by the present time domain decoding is stored in the buffer.

[0235] 4. When the fourth frame is obtained, since the fourth frame is still not a time domain stationary signal and the last frame is in the time domain encoding mode, the final encoding mode of the fourth frame is the time domain encoding mode. The audio transmitting device forms first data (f and g) of length N by combining the first N / 2 data g of the fourth frame with the last N / 2 data f of the third frame, and then performs time domain encoding on the first data. The audio receiving device performs time domain decoding on the code stream to obtain first data (f and g) of length N. The first N / 2 data f of the first data is combined with the last N / 2 data e of the third data obtained by the last time domain decoding and stored in the buffer to form the third frame (e and f). Then the last N / 2 data g of the first data obtained by the present time domain decoding is stored in the buffer.

[0236] Embodiment Two

[0237] Figure 11 An exemplary frame variation flow chart for the present application from the time domain encoding to the frequency domain encoding switching encoding mode is shown in Fig. 2. Figure 11As shown, the audio of the music type in the high-quality sound scene contains 5 audio frames (frames 0-4), each audio frame has a length of N and is divided into two parts of N / 2, wherein frame 0 is divided into 0 and 0, frame 1 is divided into a and b, frame 2 is divided into c and d, frame 3 is divided into e and f, and frame 4 is divided into g and h, so that a-h all have a length of N / 2. Frames 0-2 are not time-domain stationary signals, and frames 3-4 are time-domain stationary signals.

[0238] 1. When the first frame is acquired, the final encoding mode of the first frame is time-domain encoding, the audio sending device combines the latter N / 2 data 0 of the first frame and the former N / 2 data a of the second frame into first data (0 and a) with a length of N, and performs time-domain encoding on the first data. The audio receiving device performs time-domain decoding on the code stream to obtain the first data (0 and a) with a length of N, then combines the former N / 2 data 0 of the first data and the data 0 stored in the cache obtained by the last time-domain decoding to form the first frame (0 and 0). Then, the latter N / 2 data a of the first data is stored in the cache.

[0239] 2. When the second frame is acquired, the final encoding mode of the second frame is time-domain encoding, the audio sending device combines the latter N / 2 data b of the first frame and the former N / 2 data c of the second frame into first data (b and c) with a length of N, and performs time-domain encoding on the first data. The audio receiving device performs time-domain decoding on the code stream to obtain the first data (b and c) with a length of N, then combines the former N / 2 data b of the first data and the latter N / 2 data a of the first data stored in the cache obtained by the last time-domain decoding to form the first frame (a and b). Then, the latter N / 2 data c of the first data is stored in the cache.

[0240] 3. When the third frame is acquired, since the third frame is a time-domain stationary signal and the last frame is in a time-domain encoding state, the final encoding mode of the third frame is a time-domain encoding to frequency-domain encoding switching encoding mode, the audio sending device performs folding processing on the third frame (e and f) to obtain first data e+fr with a length of N / 2, then combines the first data and the latter N / 2 data d of the second frame into second data (d and e+fr) with a length of N, and performs time-domain encoding on the second data. The audio receiving device performs time-domain decoding on the code stream to obtain the second data (d and e+fr) with a length of N, performs unfolding processing on the latter N / 2 data e+fr of the second data to obtain third data (e+fr and er+f) with a length of N. The audio receiving device combines the former N / 2 data d of the second data and the latter N / 2 data c of the first data stored in the cache obtained by the last time-domain decoding to form the second frame (c and d). Then, the third data (e+fr and er+f) is stored in the cache.

[0241] 4、When the fourth frame is acquired, the final encoding mode of the fourth frame is frequency domain encoding mode because the fourth frame is still time domain stationary signal and the last frame is time domain encoding to frequency domain encoding switching encoding mode. The audio sending device encodes the first data (e, f, g and h) with length of 2N composed of the fourth frame and the third frame by frequency domain encoding method as shown in the formula (3). Figure 7 The audio receiving device decodes the bitstream by frequency domain decoding to obtain the second data with length of N, and then performs IMDCT transform on the second data to obtain the third data (e-fr, -er+f, g+hr and gr+h) with length of 2N. The audio receiving device superimposes the first N data (e-fr and -er+f) of the third data with the third data (e+fr and er+f) obtained by the last time domain decoding and stored in the buffer to recover the third frame (e and f). Then the last N data (g+hr and gr+h) of the third data obtained by the present frequency domain decoding is stored in the buffer.

[0242] Embodiment Three

[0243] Figure 12a And 12b An exemplary flow chart of switching encoding and decoding mode of the present application is shown in FIG. 2, which is different from the block diagram shown in FIG. 1 in that, in the frequency domain encoding to time domain encoding switching encoding mode, the audio receiving device needs to perform interpolation processing after decoding the bitstream; in the time domain encoding to frequency domain encoding switching encoding mode, the audio sending device needs to perform interpolation processing before encoding. Figure 12a 12b Figure 9 The difference between the block diagram shown in FIG. 1 and the block diagram shown in FIG. 2 is that, in the frequency domain encoding to time domain encoding switching encoding mode, the audio receiving device needs to perform interpolation processing after decoding the bitstream; in the time domain encoding to frequency domain encoding switching encoding mode, the audio sending device needs to perform interpolation processing before encoding.

[0244] Figure 13 An exemplary frame change flow chart of the present application from frequency domain encoding to time domain encoding switching encoding mode is shown in FIG. 3, which is a high sound quality scene of music type audio containing five audio frames (0-4 frames), each audio frame has a length of N and is divided into two parts of N / 2 data in front and N / 2 data in back, wherein the 0th frame is 0 and 0 respectively, the 1st frame is divided into a and b, the 2nd frame is divided into c and d, the 3rd frame is divided into e and f, and the 4th frame is divided into g and h, so the lengths of a-h are all N / 2. The 0th-2nd frames are time domain stationary signals, and the 3rd-4th frames are not time domain stationary signals. Figure 13 1、When the first frame is acquired, the final encoding mode of the first frame is frequency domain encoding mode, and the audio sending device combines the 0th frame and the 1st frame into the first data (0, 0, a and b) with length of 2N and encodes the first data by frequency domain encoding method as shown in the formula (3).

[0245] Figure 7 ​​​The method shown encodes the first data in frequency domain. The audio receiving device performs frequency domain decoding on the code stream to obtain second data with length N, and then performs IMDCT transform on the second data to obtain third data (0, 0, a+br and ar+b) with length 2N. Since there is no -1th frame data, the audio receiving device assumes the -1th frame data as 0, and then superimposes the first N data (0 and 0) of the third data and 0 to obtain the 0th frame data 0 and 0. Then the last N data (a+br and ar+b) of the third data are stored in the buffer.

[0246] 2. When the 2nd frame is acquired, the final encoding mode of the 2nd frame is frequency domain encoding mode, the audio sending device combines the 1st frame and the 2nd frame into first data (a, b, c and d) with length 2N, and then performs frequency domain encoding on the first data. Figure 7 The method shown encodes the first data in frequency domain. The audio receiving device performs frequency domain decoding on the code stream to obtain second data with length N, and then performs IMDCT transform on the second data to obtain third data (a-br, -ar+b, c+dr and cr+d) with length 2N. The audio receiving device superimposes the first N data (a-br and -ar+b) of the third data and the last N data (a+br and ar+b) of the third data obtained by the last frequency domain decoding and stored in the buffer to recover the 1st frame (a and b). Then the c+dr in the last N data of the third data obtained by the current frequency domain decoding is equally spaced interpolated from the last N / 2-D data to obtain fourth data (c+dr) i with length N / 2, and cr+d is equally spaced interpolated from the first N / 2-D data to obtain fifth data (cr+d) i with length N / 2, where the superscript i represents interpolation processing, and (c+dr) i and (cr+d) i are stored in the buffer.

[0247] 3. When the 3rd frame is acquired, since the 3rd frame is not a time domain stationary signal, and the last frame is in frequency domain encoding state, the final encoding mode of the 3rd frame is frequency domain encoding to time domain encoding switching encoding mode, the audio sending device performs folding processing on the 2nd frame c and d to obtain first data -cr+d with length N / 2, and then combines the first data and the first N / 2 data e of the 3rd frame into second data (-cr+d and e) with length N, and then performs time domain encoding on the second data. The audio receiving device performs time domain decoding on the code stream to obtain third data ((-cr+d)' and e') with length N, where the superscript'represents an additional D sample time delay introduced by QMF. First, the first N / 2 data (-cr+d)' of the third data are equally spaced interpolated from the last N / 2-D data to obtain fourth data (-cr+d) iThe fourth data is processed by odd symmetry expansion to obtain fifth data (c-dr) of length N i and (-cr+d) i ), and then the fifth data (c-dr) of length N i and (-cr+d) i ) is superimposed with the (c+dr) of length N i and (cr+d) of length N i ) obtained by the previous frequency domain decoding and stored in the buffer to recover the second frame data (c i and d i ). The last N / 2 data e' of the third data obtained by the present time domain decoding is stored in the buffer.

[0248] 4. When the fourth frame is obtained, since the fourth frame is still not a time domain stationary signal, and the previous frame is a frequency domain encoding to time domain encoding switching encoding mode, the final encoding mode of the fourth frame is time domain encoding mode, and the audio sending device groups the first N / 2 data g of the fourth frame and the last N / 2 data f of the third frame to form first data (f and g) of length N, and then performs time domain encoding on the first data. The audio receiving device performs time domain decoding on the code stream to obtain second data (f' and g') of length N, and groups the first N / 2 data f' of the second data and the last N / 2 data e' of the third data obtained by the previous time domain decoding and stored in the buffer to form the third frame (e' and f'). Then the last N / 2 data g' of the second data obtained by the present time domain decoding is stored in the buffer.

[0249] From the data output by the audio receiving device, the 0th frame 0 and 0 are output when the first frame is decoded, the 1st frame a and b are output when the second frame is decoded, the 2nd frame c i and d i are output when the third frame is decoded, and the 3rd frame e' and f' are output when the fourth frame is decoded. Among them, the output when the third frame is decoded is the result of interpolation from N / 2-D data to N / 2 data, so the second frame is data stretching processing, and the actual performance is audio slow play processing. Since the value of D is much smaller than N / 2 in actual situation, the audio slow play will not be perceived by people.

[0250] Embodiment four

[0251] Figure 14 This is an exemplary frame change flow chart of the present application from time domain encoding to frequency domain encoding switching encoding mode, as Figure 14As shown, the audio of the music type in the high-quality sound scene contains 5 audio frames (frames 0-4), each audio frame has a length of N and is divided into a first N / 2 data and a second N / 2 data, wherein the first frame is divided into 0 and 0, the second frame is divided into a and b, the third frame is divided into c and d, the fourth frame is divided into e and f, and the fifth frame is divided into g and h, so that a, b, c, d, e, f, g and h each have a length of N / 2. The first, second and third frames are not time-domain stationary signals, and the fourth and fifth frames are time-domain stationary signals.

[0252] 1. When the first frame is acquired, the final encoding mode of the first frame is time-domain encoding, the audio sending device combines the second N / 2 data 0 of the first frame and the first N / 2 data a of the second frame into first data (0 and a) with a length of N, and performs time-domain encoding on the first data. The audio receiving device performs time-domain decoding on the code stream to obtain second data (0' and a') with a length of N, and then combines the first N / 2 data 0' of the first data and the data 0' stored in the buffer obtained by the last time-domain decoding to obtain the first frame (0' and 0'). Then, the second N / 2 data a' of the first data is stored in the buffer.

[0253] 2. When the second frame is acquired, the final encoding mode of the second frame is time-domain encoding, the audio sending device combines the third N / 2 data b of the first frame and the first N / 2 data c of the second frame into first data (b and c) with a length of N, and performs time-domain encoding on the first data. The audio receiving device performs time-domain decoding on the code stream to obtain first data (b' and c') with a length of N, and then combines the first N / 2 data b' of the first data and the second N / 2 data a' of the first data stored in the buffer obtained by the last time-domain decoding to obtain the first frame (a' and b'), which is data delayed by N+D samples. Then, the second N / 2 data c' of the first data is stored in the buffer.

[0254] 3. When the third frame is acquired, since the third frame is a time-domain stationary signal and the last frame is in a time-domain encoding state, the final encoding mode of the third frame is a time-domain encoding to frequency-domain encoding switching encoding mode, the audio sending device performs folding processing on the third frame (e and f) to obtain first data e+fr with a length of N / 2, then combines the N / 2 data of the first data e+fr with the D data of er+f obtained by even symmetry unfolding to obtain second data with a length of N / 2+D, and equally-spaced interpolates the second data into third data (e+fr) with a length of N / 2. i The audio sending device combines the third data (e+fr) i with the second N / 2 data d of the second frame to obtain fourth data (d and (e+fr) i ) with a length of N, and performs time-domain encoding on the fourth data. The audio receiving device performs time-domain decoding on the code stream to obtain fourth data (d' and (e+fr) i), the last N / 2 data of the fourth data (e+fr) i , and the fifth data ((e+fr) i and (er+f) i ) is obtained. The audio receiving device forms the second frame (c' and d') by combining the first N / 2 data d' of the fourth data and the last N / 2 data c' of the first data obtained by the last time domain decoding and stored in the buffer. Then the fifth data ((e+fr) i and (er+f) i ) is stored in the buffer.

[0255] 4. When the fourth frame is obtained, since the fourth frame is still a time domain stationary signal, and the last frame is switched from the time domain encoding to the frequency domain encoding, the final encoding mode of the fourth frame is the frequency domain encoding mode. The audio sending device forms the first data (e, f, g and h) with a length of 2N by combining the fourth frame and the third frame, and encodes the first data in the frequency domain by using the method shown in Figure 7 . The audio receiving device decodes the code stream in the frequency domain to obtain the second data with a length of N, and then performs the IMDCT transformation on the second data to obtain the third data (e-fr, -er+f, g+hr and gr+h) with a length of 2N. The audio receiving device combines e-fr in the first N data of the third data with the last D data to form the fourth data with a length of N / 2+D, and obtains the fifth data (e-fr) i by performing the equal-interval interpolation on the fourth data. The audio receiving device combines -er+f in the first N data of the third data with the first D data to form the sixth data with a length of N / 2+D, and obtains the seventh data (-e+fr) i by performing the equal-interval interpolation on the sixth data. The audio receiving device superimposes ((e-fr) i and (-e+fr) i ) with the fifth data ((e+fr) i and (er+f) i ) obtained by the last time domain decoding and stored in the buffer to recover the third frame (e i and f i ). Then the last N data (g+hr and gr+h) of the third data obtained by the present frequency domain decoding is stored in the buffer.

[0256] From the data output by the audio receiving device, the first frame 0' and 0' are output when the first frame is decoded, the second frame a' and b' are output when the second frame is decoded, the third frame c' and d' are output when the third frame is decoded, and the fourth frame e i and f iThe output of decoding the fourth frame is the result of interpolation from N / 2+D data to N / 2 data, and the third frame is data scaling processing, which is actually audio fast-forward processing. Since the value of D is much smaller than N / 2 in actual situations, the audio fast-forward processing cannot be perceived by people.

[0257] It should be noted that there are many ways of interpolation, such as linear interpolation, cubic spline interpolation, polynomial interpolation, etc., which are not limited in the present application.

[0258] Embodiment Five

[0259] In the frequency domain encoding mode, the time domain encoding to frequency domain encoding switching encoding mode, and the frequency domain encoding to time domain encoding switching encoding mode, the audio sending device in the high-quality scene of the music type of audio in the above-mentioned embodiments one to four has a step of windowing the audio frame before the folding processing, and the audio receiving device has a step of de-windowing the audio frame after the unfolding processing.

[0260] If it is a normal window, the audio can be encoded and decoded according to the methods of the above-mentioned embodiments one to four.

[0261] If it is a low-delay window, the left shift operation needs to be performed on the multiple samples contained in the audio in the windowing processing, for example, as shown in FIG. 6, left shift M samples; the left shift operation needs to be performed on the multiple samples contained in the audio in the de-windowing processing, for example, as shown in FIG. 7, left shift M samples. Other operations can be performed according to the methods of the above-mentioned embodiments one to four. Left shift M samples means moving M samples to the latest data, that is, reducing the delay by M samples. Figure 6b Figure 6c

[0262] Figure 15 The structure schematic diagram of the audio encoding device embodiment of the present application is shown in FIG. 15. The device can be applied to the audio sending device in the above-mentioned embodiments. The encoding device of the present embodiment can include a determination module 1501 and an encoding module 1502. Wherein, Figure 15

[0263] ​​​The determination module 1501 is configured to determine a signal type and a use scenario of a current audio frame to be encoded, the use scenario including a low-delay scenario or a high-audio-quality scenario; determine an initial encoding mode of the current audio frame according to the signal type of the current audio frame and the use scenario of the current audio frame, the initial encoding mode including a time-domain encoding mode or a frequency-domain encoding mode; determine a final encoding mode of the current audio frame according to the initial encoding mode of the current audio frame and a final encoding mode of a previous audio frame, the final encoding mode including the time-domain encoding mode, the frequency-domain encoding mode, a time-domain-to-frequency-domain encoding switching mode, or a frequency-domain-to-time-domain encoding switching mode; and the encoding module 1502 is configured to encode the current audio frame according to the final encoding mode of the current audio frame.

[0264] In a possible implementation, the determination module 1501 is specifically configured to determine that the initial encoding mode of the current audio frame is the time-domain encoding mode when the signal type of the current audio frame is a music type and the use scenario is the low-delay scenario; or determine whether the current audio frame is a time-domain stationary signal when the signal type of the current audio frame is the music type and the use scenario is the high-audio-quality scenario; determine that the initial encoding mode of the current audio frame is the frequency-domain encoding mode if the current audio frame is the time-domain stationary signal; or determine that the initial encoding mode of the current audio frame is the time-domain encoding mode if the current audio frame is not the time-domain stationary signal.

[0265] In a possible implementation, the determination module 1501 is specifically configured to split the audio frame into a plurality of audio subframes; calculate energy variances of the plurality of audio subframes; filter the audio frame by using a linear prediction coding (LPC) filter to obtain energy of a filtered audio frame when the energy variances of the plurality of audio subframes are greater than a first set threshold, and determine whether the audio frame is the time-domain stationary signal according to the energy of the filtered audio frame; and transform the audio frame by using an improved discrete cosine transform (MDCT) to obtain energy variances of each frequency point of a transformed audio frame when the energy variances of the plurality of audio subframes are less than the first set threshold, and determine whether the audio frame is the time-domain stationary signal according to the energy variances of each frequency point of the transformed audio frame.

[0266] In a possible implementation, the determination module 1501 is specifically configured to determine that the audio frame is the time-domain stationary signal if a ratio of the energy of the filtered audio frame to energy of the audio frame is greater than a second set threshold; or determine that the audio frame is not the time-domain stationary signal if the ratio of the energy of the filtered audio frame to the energy of the audio frame is less than the second set threshold.

[0267] In a possible implementation, the determining module 1501 is specifically configured to: if the energy variance of each frequency point of the transformed audio frame is greater than a third preset threshold, determining that the audio frame is the time-domain stationary signal; or if the energy variance of each frequency point of the transformed audio frame is less than the third preset threshold, determining that the audio frame is not the time-domain stationary signal.

[0268] In a possible implementation, the determining module 1501 is specifically configured to: when the initial encoding mode of the current audio frame and the final encoding mode of the previous audio frame are both frequency-domain encoding modes, determining that the final encoding mode of the current audio frame is a frequency-domain encoding mode; or when the initial encoding mode of the current audio frame is a frequency-domain encoding mode and the final encoding mode of the previous audio frame is a time-domain encoding mode, determining that the final encoding mode of the current audio frame is a time-domain-to-frequency-domain switching encoding mode; or when the initial encoding mode of the current audio frame is a frequency-domain encoding mode and the final encoding mode of the previous audio frame is a time-domain-to-frequency-domain switching encoding mode, determining that the final encoding mode of the current audio frame is a frequency-domain encoding mode; or when the initial encoding mode of the current audio frame is a frequency-domain encoding mode and the final encoding mode of the previous audio frame is a frequency-domain-to-time-domain switching encoding mode, determining that the final encoding mode of the current audio frame is a time-domain-to-frequency-domain switching encoding mode; or when the initial encoding mode of the current audio frame is a time-domain encoding mode and the final encoding mode of the previous audio frame is a frequency-domain encoding mode, determining that the final encoding mode of the current audio frame is a frequency-domain-to-time-domain switching encoding mode; or when the initial encoding mode of the current audio frame is a time-domain encoding mode and the final encoding mode of the previous audio frame is a time-domain-to-frequency-domain switching encoding mode, determining that the final encoding mode of the current audio frame is a frequency-domain-to-time-domain switching encoding mode; or when the initial encoding mode of the current audio frame is a time-domain encoding mode and the final encoding mode of the previous audio frame is a frequency-domain-to-time-domain switching encoding mode, determining that the final encoding mode of the current audio frame is a time-domain encoding mode.

[0269] In a possible implementation, when the final encoding mode is the time-domain encoding mode, the encoding module 1502 is specifically configured to: splice the last N / 2 sample data of the previous audio frame and the first N / 2 sample data of the current audio frame to obtain first data with a length of N, N representing the length of an audio frame; and perform time-domain encoding on the first data.

[0270] In a possible implementation, when the final encoding mode is the frequency domain encoding mode, the encoding module 1502 is specifically configured to splice the N sample point data of the previous audio frame and the N sample point data of the current audio frame to obtain second data with a length of 2N, N representing the length of an audio frame; perform windowing processing on the second data to obtain third data with a length of 2N; and perform frequency domain encoding on the third data.

[0271] In a possible implementation, when the final encoding mode is the frequency domain encoding mode, the encoding module 1502 is specifically configured to splice the N sample point data of the previous audio frame and the N sample point data of the current audio frame to obtain second data with a length of 2N, N representing the length of an audio frame; perform windowing processing on the second data to obtain third data with a length of 2N; and perform frequency domain encoding on the third data.

[0272] In a possible implementation, when the final encoding mode is the frequency domain encoding mode, the encoding module 1502 is specifically configured to splice the N sample point data of the previous audio frame and the N sample point data of the current audio frame to obtain second data with a length of 2N, N representing the length of an audio frame; perform windowing processing on the second data to obtain third data with a length of 2N; and perform frequency domain encoding on the third data.

[0273] In a possible implementation, when the final encoding mode is the time-domain encoding to frequency-domain encoding switching encoding mode, and the total time delay of the time-domain encoding and decoding is N+D sample data, the encoding module 1502 is specifically configured to perform windowing processing on the N sample data of the current audio frame to obtain tenth data with a length of N; perform even symmetry folding processing on the tenth data with a center position as an even symmetry point to obtain eleventh data with a length of N / 2; perform even symmetry unfolding processing on the eleventh data with the center position as the even symmetry point to obtain twelfth data with a length of N, the first N / 2 sample data of the twelfth data is the eleventh data, and the last N / 2 sample data of the twelfth data is thirteenth data with a length of N / 2; splice the first D sample data of the eleventh data and the thirteenth data to obtain fourteenth data with a length of N / 2+D, D representing a time delay of QMF in the time-domain encoding mode; perform interpolation processing on the fourteenth data to obtain fifteenth data with a length of N / 2; splice the last N / 2 sample data of the previous audio frame and the fifteenth data to obtain sixteenth data with a length of N; and perform time-domain encoding on the sixteenth data.

[0274] In a possible implementation, the current audio frame is a pulse code modulation (PCM) signal, or the current audio frame is a signal obtained by performing quadrature mirror filtering (QMF) on the PCM signal, or the current audio frame is a residual signal obtained by performing inverse LPC filtering on the PCM signal.

[0275] The apparatus of the embodiment can be used to execute the method of the embodiment. Figures 3 to 14 The technical solutions of the method embodiment are similar in principle and technical effects to the apparatus embodiment, and are not described herein again.

[0276] Figure 16 FIG. 1 shows a structural schematic diagram of an audio decoding apparatus embodiment of the present application. Figure 16 The apparatus can be applied to the audio receiving device in the above embodiment. The encoding apparatus of the embodiment can include an obtaining module 1601 and a decoding module 1602. The obtaining module 1601 is configured to obtain a bitstream, and the decoding module 1602 is configured to parse the bitstream to obtain bitstream data to be decoded and indication information of a decoding mode, the decoding mode including a time-domain decoding mode, a frequency-domain decoding mode, a time-domain decoding to frequency-domain decoding switching decoding mode, or a frequency-domain decoding to time-domain decoding switching decoding mode; and decode the bitstream data according to the decoding mode.

[0277] The obtaining module 1601 is configured to obtain a bitstream, and the decoding module 1602 is configured to parse the bitstream to obtain bitstream data to be decoded and indication information of a decoding mode, the decoding mode including a time-domain decoding mode, a frequency-domain decoding mode, a time-domain decoding to frequency-domain decoding switching decoding mode, or a frequency-domain decoding to time-domain decoding switching decoding mode; and decode the bitstream data according to the decoding mode.

[0278] In a possible implementation, when the decoding manner is the time domain decoding manner, the decoding module 1602 is specifically configured to perform time domain decoding on the code stream data to obtain a seventeenth data with a length of N, N representing a length of an audio frame, N / 2 sample data at the back of the seventeenth data being N / 2 sample data at the front of a current audio frame, and N / 2 sample data at the front of the seventeenth data being N / 2 sample data at the back of a previous audio frame of the current audio frame; splice the N / 2 sample data at the front of the previous audio frame and the N / 2 sample data at the front of the seventeenth data to obtain data of the previous audio frame; and store the N / 2 sample data at the back of the seventeenth data in a buffer.

[0279] In a possible implementation, when the decoding manner is the frequency domain decoding manner, and a total time delay of time domain coding and decoding is N+D sample data, and a decoding manner of a previous audio frame is a time domain decoding to frequency domain decoding switching decoding manner, the decoding module 1602 is specifically configured to perform frequency domain decoding on the code stream data to obtain an eighteenth data with a length of 2N, N representing a length of an audio frame; splice N / 2 sample data at the front of N sample data at the front of the eighteenth data and D sample data after the N / 2 sample data to obtain a nineteenth data with a length of N / 2+D, D representing a time delay of a quadrature mirror filter QMF in the time domain coding manner; perform interpolation processing on the nineteenth data to obtain a twentieth data with a length of N / 2; splice N / 2 sample data at the back of N sample data at the back of the eighteenth data and D sample data before the N / 2 sample data to obtain a twenty-first data with a length of N / 2+D; perform interpolation processing on the twenty-first data to obtain a twenty-second data with a length of N / 2; splice the twentieth data and the twenty-second data to obtain a twenty-third data with a length of N; perform windowing overlap processing on the twenty-third data and N sample data in a buffer obtained by previous frequency domain decoding to obtain data of the previous audio frame; and store N sample data at the back of the eighteenth data in the buffer.

[0280] In a possible implementation, when the decoding manner is the frequency domain decoding manner, and the total time delay of time domain coding and decoding is N+D sample data, N represents the length of an audio frame, D represents the time delay of QMF in the time domain coding manner, and the decoding manner of a previous audio frame is the frequency domain decoding manner; or when the decoding manner is the frequency domain decoding manner, and the total time delay of time domain coding and decoding is N sample data, the decoding module 1602 is specifically configured to perform frequency domain decoding on the code stream data to obtain twenty-fourth data with a length of 2N; perform windowing overlap processing on the first N sample data of the twenty-fourth data and the last N sample data of twenty-fifth data with a length of 2N obtained by previous frequency domain decoding to obtain data of a previous audio frame; and store the last N sample data of the twenty-fourth data in a buffer.

[0281] In a possible implementation, when the decoding manner is the frequency domain decoding manner, and the total time delay of time domain coding and decoding is N+D sample data, N represents the length of an audio frame, D represents the time delay of QMF in the time domain coding manner, and the decoding manner of a previous audio frame is the frequency domain decoding manner; or when the decoding manner is the frequency domain decoding manner, and the total time delay of time domain coding and decoding is N sample data, the decoding module 1602 is specifically configured to perform frequency domain decoding on the code stream data to obtain twenty-fourth data with a length of 2N; perform windowing overlap processing on the first N sample data of the twenty-fourth data and the last N sample data of twenty-fifth data with a length of 2N obtained by previous frequency domain decoding to obtain data of a previous audio frame; and store the last N sample data of the twenty-fourth data in a buffer.

[0282] In a possible implementation, when the decoding manner is switched from the frequency domain decoding to the time domain decoding, and the total time delay of the time domain coding is N+D sample data, N represents the length of an audio frame, and D represents the time delay of the QMF in the time domain coding manner, the decoding module 1602 is specifically configured to perform time domain decoding on the code stream data to obtain a twenty-ninth data with a length of N; perform interpolation processing on the last N / 2-D sample data of the first N / 2 sample data of the twenty-ninth data to obtain a thirtieth data with a length of N / 2; perform odd symmetric expansion processing on the thirtieth data with a center position as an odd symmetric point to obtain a thirty-first data with a length of N; perform interpolation processing on the first N / 2-D sample data of the last N sample data of a thirty-second data with a length of 2N obtained by last frequency domain decoding to obtain a thirty-third data with a length of N / 2; perform interpolation processing on the last N / 2-D sample data of the last N sample data of the thirty-second data to obtain a thirty-fourth data with a length of N / 2; splice the thirty-third data and the thirty-fourth data to obtain a thirty-fifth data with a length of N; perform de-windowing overlap processing on the thirty-first data and the thirty-fifth data to obtain data of a last audio frame with a length of N; and store the last N / 2 sample data of the twenty-ninth data in a buffer.

[0283] In a possible implementation, when the decoding manner is switched from the time domain decoding to the frequency domain decoding, the decoding module 1602 is specifically configured to perform time domain decoding on the code stream data to obtain a thirty-sixth data with a length of N, N representing the length of an audio frame; splice the last N / 2 sample data of a thirty-seventh data with a length of N obtained by last time domain decoding and the first N / 2 sample data of the thirty-sixth data to obtain data of a last audio frame with a length of N; perform even symmetric expansion processing on the last N / 2 sample data of the thirty-sixth data with a center position as an even symmetric point to obtain a thirty-eighth data with a length of N; and store the thirty-eighth data in a buffer.

[0284] In a possible implementation, the last audio frame is a pulse code modulation (PCM) signal; or, the last audio frame is a PCM signal obtained by a quadrature mirror filter (QMF); or, the last audio frame is a PCM signal obtained by an LPC filter.

[0285] The apparatus of the embodiment can be used to execute the method. Figures 3 to 14 The technical solutions of the method embodiments shown in the drawings have similar implementation principles and technical effects, which will not be described herein.

[0286] In the implementation process, each step of the above method embodiments can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware coding executed by the processor to complete, or executed by a combination of hardware and software modules in the coding processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the storage, and the processor reads the information in the storage, and combines the hardware to complete the steps of the above method.

[0287] The memory mentioned in each of the above embodiments can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct Rambus RAM (DR RAM). It should be noted that the memory of the system and method described herein is intended to include, but not be limited to, these and any other suitable types of memory.

[0288] Those of ordinary skill in the art can be aware that, in combination with the embodiments disclosed in this paper, the units and algorithm steps of each example can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0289] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0290] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0291] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0292] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit.

[0293] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to make a computer device (personal computer, server, or network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various program codes that can be stored in the medium.

[0294] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An audio encoding method, characterized by, The method comprises the following steps: determining the signal type and use scenario of a current audio frame to be encoded, wherein the use scenario comprises a low-delay scenario or a high-audio-quality scenario; determining an initial encoding mode of the current audio frame according to the signal type of the current audio frame and the use scenario of the current audio frame, wherein the initial encoding mode comprises a time-domain encoding mode or a frequency-domain encoding mode; determining a final encoding mode of the current audio frame according to the initial encoding mode of the current audio frame and the final encoding mode of a previous audio frame, wherein the final encoding mode comprises the time-domain encoding mode, the frequency-domain encoding mode, a time-domain-to-frequency-domain encoding switching mode or a frequency-domain-to-time-domain encoding switching mode; encoding the current audio frame according to the final encoding mode of the current audio frame.

2. The method of claim 1, wherein, The step of determining the initial encoding mode of the current audio frame according to the signal type of the current audio frame and the use scenario of the current audio frame comprises: when the signal type of the current audio frame is a music type and the use scenario is the low-delay scenario, determining that the initial encoding mode of the current audio frame is the time-domain encoding mode; or when the signal type of the current audio frame is the music type and the use scenario is the high-audio-quality scenario, determining whether the current audio frame is a time-domain stationary signal; if the current audio frame is the time-domain stationary signal, determining that the initial encoding mode of the current audio frame is the frequency-domain encoding mode; or if the current audio frame is not the time-domain stationary signal, determining that the initial encoding mode of the current audio frame is the time-domain encoding mode.

3. The method of claim 2, wherein, The step of determining whether the current audio frame is the time-domain stationary signal comprises: splitting the current audio frame into a plurality of audio subframes; calculating the energy variance of the plurality of audio subframes; when the energy variance of the plurality of audio subframes is greater than a first set threshold, filtering the audio frame by using a linear predictive coding (LPC) filter to obtain the energy of the filtered audio frame, and determining whether the audio frame is the time-domain stationary signal according to the energy of the filtered audio frame; when the energy variance of the plurality of audio subframes is less than the first set threshold, transforming the audio frame by using a modified discrete cosine transform (MDCT) to obtain the energy variance of each frequency point of the transformed audio frame, and determining whether the audio frame is the time-domain stationary signal according to the energy variance of each frequency point of the transformed audio frame.

4. The method of claim 3, wherein, The step of determining whether the audio frame is the time-domain stationary signal according to the energy of the filtered audio frame comprises: if the ratio of the energy of the filtered audio frame to the energy of the audio frame is greater than a second set threshold, the audio frame is the time-domain stationary signal; if the ratio of the energy of the filtered audio frame to the energy of the audio frame is less than the second set threshold, the audio frame is not the time-domain stationary signal.

5. The method of claim 3, wherein, The step of determining whether the audio frame is the time-domain stationary signal according to the energy variance of each frequency point of the transformed audio frame comprises: If the energy variance of each frequency point of the transformed audio frame is greater than a third set threshold, the audio frame is the time domain stationary signal; if the energy variance of each frequency point of the transformed audio frame is less than the third set threshold, the audio frame is not the time domain stationary signal.

6. The method according to any one of claims 1-5, characterized in that, The final encoding mode of the current audio frame is determined according to the initial encoding mode of the current audio frame and the final encoding mode of the previous audio frame, comprising: When the initial encoding mode of the current audio frame and the final encoding mode of the previous audio frame are both frequency domain encoding modes, the final encoding mode of the current audio frame is determined as a frequency domain encoding mode; or, When the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a time domain encoding mode, the final encoding mode of the current audio frame is determined as a time domain to frequency domain switching encoding mode; or, When the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a time domain to frequency domain switching encoding mode, the final encoding mode of the current audio frame is determined as a frequency domain encoding mode; or, When the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain to time domain switching encoding mode, the final encoding mode of the current audio frame is determined as a time domain to frequency domain switching encoding mode; or, When the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain encoding mode, the final encoding mode of the current audio frame is determined as a frequency domain to time domain switching encoding mode; or, When the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a time domain to frequency domain switching encoding mode, the final encoding mode of the current audio frame is determined as a frequency domain to time domain switching encoding mode; or, When the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain to time domain switching encoding mode, the final encoding mode of the current audio frame is determined as a time domain encoding mode. When the final encoding mode is the time domain encoding mode, the encoding of the current audio frame according to the final encoding mode comprises:

7. The method according to any one of claims 1 to 6, characterized in that, The last N / 2 sample data of the previous audio frame and the first N / 2 sample data of the current audio frame are spliced to obtain first data with a length of N, N representing the length of the audio frame; The first data is time domain encoded. When the final encoding mode is the frequency domain encoding mode, the encoding of the current audio frame according to the final encoding mode comprises:

8. The method according to any one of claims 1-6, characterized in that, ​ Splicing the N-sample data of the previous audio frame and the N-sample data of the current audio frame to obtain second data with a length of 2N, N representing the length of an audio frame; Performing windowing processing on the second data to obtain third data with a length of 2N; Performing frequency domain coding on the third data.

9. The method according to any one of claims 1-6, characterized in that, When the final coding mode is a switch from the frequency domain coding to the time domain coding, the encoding of the current audio frame according to the final coding mode comprises: Performing windowing processing on the N-sample data of the previous audio frame to obtain fourth data with a length of N, N representing the length of an audio frame; Performing odd symmetry folding processing with a center position as an odd symmetry point on the fourth data to obtain fifth data with a length of N / 2; Splicing the fifth data and the first N / 2 sample data of the current audio frame to obtain sixth data with a length of N; Performing time domain coding on the sixth data.

10. The method according to any one of claims 1-6, characterized in that, When the final coding mode is a switch from the time domain coding to the frequency domain coding, and the total time delay of the time domain coding and decoding is N-sample data, the encoding of the current audio frame according to the final coding mode comprises: Performing windowing processing on the N-sample data of the current audio frame to obtain seventh data with a length of N, N representing the length of an audio frame; Performing even symmetry folding processing with a center position as an even symmetry point on the seventh data to obtain eighth data with a length of N / 2; Splicing the last N / 2 sample data of the previous audio frame and the eighth data to obtain ninth data with a length of N; Performing time domain coding on the ninth data.

11. The method according to any one of claims 1-6, characterized in that, When the final coding mode is a switch from the time domain coding to the frequency domain coding, and the total time delay of the time domain coding and decoding is N+D-sample data, the encoding of the current audio frame according to the final coding mode comprises: Performing windowing processing on the N-sample data of the current audio frame to obtain tenth data with a length of N; Performing even symmetry folding processing with a center position as an even symmetry point on the tenth data to obtain eleventh data with a length of N / 2, N representing the length of an audio frame; Performing even symmetry unfolding processing with a center position as an even symmetry point on the eleventh data to obtain twelfth data with a length of N, the first N / 2 sample data of the twelfth data being the eleventh data, and the last N / 2 sample data of the twelfth data being thirteenth data with a length of N / 2; Splicing the eleventh data and the first D-sample data of the thirteenth data to obtain fourteenth data with a length of N / 2+D, D representing the time delay of QMF in the time domain coding mode; Performing interpolation processing on the fourteenth data to obtain fifteenth data with a length of N / 2; Splicing the last N / 2 sample data of the previous audio frame and the fifteenth data to obtain sixteenth data with a length of N; Performing time domain coding on the sixteenth data.

12. The method according to any one of claims 1-11, characterized in that, The current audio frame is a pulse code modulation (PCM) signal; or The current audio frame is a signal obtained by performing quadrature mirror filtering (QMF) on a PCM signal; or The current audio frame is a residual signal obtained by performing inverse LPC filtering on a PCM signal.

13. An audio decoding method, characterized by, Comprise: Obtaining a code stream; Parsing the code stream to obtain code stream data to be decoded and indication information of a decoding mode, the decoding mode including a time domain decoding mode, a frequency domain decoding mode, a time domain decoding to frequency domain decoding switching decoding mode, or a frequency domain decoding to time domain decoding switching decoding mode, the decoding mode being determined according to an initial encoding mode of a current audio frame and a final encoding mode of a previous audio frame; the initial encoding mode being determined according to a signal type of the current audio frame and a use scenario of the current audio frame, the initial encoding mode including a time domain encoding mode or a frequency domain encoding mode, the use scenario including a low delay scenario or a high sound quality scenario; Decoding the code stream data according to the decoding mode.

14. The method of claim 13, wherein, When the decoding mode is the time domain decoding mode, the decoding the code stream data according to the decoding mode includes: Performing time domain decoding on the code stream data to obtain a seventeenth data with a length of N, N representing a length of an audio frame, back N / 2 sample point data of the seventeenth data being front N / 2 sample point data of the current audio frame, and front N / 2 sample point data of the seventeenth data being back N / 2 sample point data of the previous audio frame of the current audio frame; Splicing the front N / 2 sample point data of the previous audio frame and the front N / 2 sample point data of the seventeenth data to obtain data of the previous audio frame; Storing the back N / 2 sample point data of the seventeenth data in a buffer.

15. The method of claim 13, wherein, When the decoding mode is the frequency domain decoding mode, a total time delay of time domain coding and decoding is N+D sample point data, and a decoding mode of a previous audio frame is the time domain decoding to frequency domain decoding switching decoding mode, the decoding the code stream data according to the decoding mode includes: Performing frequency domain decoding on the code stream data to obtain an eighteenth data with a length of 2N, N representing a length of an audio frame; Splicing front N / 2 sample point data of the front N sample point data of the eighteenth data and D sample point data after the front N / 2 sample point data to obtain a nineteenth data with a length of N / 2+D, D representing a time delay of a quadrature mirror filter QMF in the time domain encoding mode; Performing interpolation processing on the nineteenth data to obtain a twentieth data with a length of N / 2; Splicing back N / 2 sample point data of the front N sample point data of the eighteenth data and D sample point data before the back N / 2 sample point data to obtain a twenty-first data with a length of N / 2+D; Performing interpolation processing on the twenty-first data to obtain a twenty-second data with a length of N / 2; Splicing the twentieth data and the twenty-second data to obtain a twenty-third data with a length of N; Performing windowing overlap processing on the twenty-third data and N sample point data in a buffer obtained by last frequency domain decoding to obtain data of the previous audio frame; Storing back N sample point data of the eighteenth data in the buffer.

16. The method of claim 13, wherein, When the decoding mode is the frequency domain decoding mode, and a total time delay of time domain coding and decoding is N+D sample data, N represents a length of an audio frame, D represents a time delay of QMF in the time domain coding mode, and a decoding mode of a previous audio frame is the frequency domain decoding mode; or when the decoding mode is the frequency domain decoding mode, and the total time delay of time domain coding and decoding is N sample data, the decoding of the code stream data according to the decoding mode comprises: frequency domain decoding of the code stream data to obtain twenty-fourth data with a length of 2N; de-windowing overlap processing of a first N sample data of the twenty-fourth data and a last N sample data of twenty-fifth data with a length of 2N obtained by previous frequency domain decoding to obtain data of a previous audio frame; storage of a last N sample data of the twenty-fourth data in a buffer.

17. The method of claim 13, wherein, When the decoding mode is switching from the frequency domain decoding mode to the time domain decoding mode, and a total time delay of time domain coding and decoding is N sample data, N represents a length of an audio frame, the decoding of the code stream data according to the decoding mode comprises: time domain decoding of the code stream data to obtain twenty-sixth data with a length of N; odd symmetry expansion processing of a first N / 2 sample data of the twenty-sixth data with a center position as an odd symmetry point to obtain twenty-seventh data with a length of N; de-windowing overlap processing of the twenty-seventh data and a last N sample data of twenty-eighth data with a length of 2N obtained by previous frequency domain decoding to obtain data of a previous audio frame with a length of N; storage of a last N / 2 sample data of the twenty-sixth data in a buffer.

18. The method of claim 13, wherein, When the decoding mode is switching from the frequency domain decoding mode to the time domain decoding mode, and a total time delay of time domain coding and decoding is N+D sample data, N represents a length of an audio frame, and D represents a time delay of QMF in the time domain coding mode, the decoding of the code stream data according to the decoding mode comprises: time domain decoding of the code stream data to obtain twenty-ninth data with a length of N; interpolation processing of a last N / 2-D sample data of a first N / 2 sample data of the twenty-ninth data to obtain thirtieth data with a length of N / 2; odd symmetry expansion processing of the thirtieth data with a center position as an odd symmetry point to obtain thirty-first data with a length of N; interpolation processing of a first N / 2-D sample data of a last N sample data of thirty-second data with a length of 2N obtained by previous frequency domain decoding to obtain thirty-third data with a length of N / 2; interpolation processing of a last N / 2-D sample data of a last N sample data of the thirty-second data to obtain thirty-fourth data with a length of N / 2; splicing of the thirty-third data and the thirty-fourth data to obtain thirty-fifth data with a length of N; de-windowing overlap processing of the thirty-first data and the thirty-fifth data to obtain data of a previous audio frame with a length of N; storage of a last N / 2 sample data of the twenty-ninth data in a buffer.

19. The method of claim 13, wherein, When the decoding mode is the time-domain decoding to frequency-domain decoding switching decoding mode, the decoding of the code stream data according to the decoding mode comprises: time-domain decoding of the code stream data to obtain thirty-sixth data with a length of N, N representing a length of an audio frame; splicing last N / 2 sample data of thirty-seventh data with a length of N obtained by previous time-domain decoding and first N / 2 sample data of the thirty-sixth data to obtain data of a previous audio frame with a length of N; performing even symmetry expansion processing of the last N / 2 sample data of the thirty-sixth data with a center position as an even symmetry point to obtain thirty-eighth data with a length of N; storing the thirty-eighth data in a cache.

20. The method of any one of claims 13-19, wherein, The previous audio frame is a pulse code modulation (PCM) signal; or The previous audio frame is a PCM signal obtained by quadrature mirror filtering (QMF); or The previous audio frame is a PCM signal obtained by linear predictive coding (LPC) filtering.

21. An audio encoding apparatus, characterized by comprising: Comprise: a determining module configured to determine a signal type and a use scenario of a current audio frame to be encoded, the use scenario comprising a low-delay scenario or a high-audio-quality scenario; determine an initial encoding mode of the current audio frame according to the signal type of the current audio frame and the use scenario of the current audio frame, the initial encoding mode comprising a time-domain encoding mode or a frequency-domain encoding mode; determine a final encoding mode of the current audio frame according to the initial encoding mode of the current audio frame and a final encoding mode of a previous audio frame, the final encoding mode comprising a time-domain encoding mode, a frequency-domain encoding mode, a time-domain encoding to frequency-domain encoding switching encoding mode, or a frequency-domain encoding to time-domain encoding switching encoding mode; an encoding module configured to encode the current audio frame according to the final encoding mode of the current audio frame.

22. The apparatus of claim 21, wherein, The determining module is specifically configured to, when the signal type of the current audio frame is a music type and the use scenario is a low-delay scenario, determine that the initial encoding mode of the current audio frame is a time-domain encoding mode; or, when the signal type of the current audio frame is a music type and the use scenario is a high-audio-quality scenario, determine whether the current audio frame is a time-domain stationary signal. If the current audio frame is a time-domain stationary signal, determine that the initial encoding mode of the current audio frame is a frequency-domain encoding mode; or, if the current audio frame is not a time-domain stationary signal, determine that the initial encoding mode of the current audio frame is a time-domain encoding mode.

23. The apparatus of claim 22, wherein, The determining module is specifically configured to split the audio frame into a plurality of audio sub-frames; calculate energy variances of the plurality of audio sub-frames; when the energy variances of the plurality of audio sub-frames are greater than a first set threshold, filter the audio frame by using a linear predictive coding (LPC) filter to obtain an energy of a filtered audio frame, and determine whether the audio frame is the time-domain stationary signal according to the energy of the filtered audio frame; and when the energy variances of the plurality of audio sub-frames are less than the first set threshold, transform the audio frame by using a modified discrete cosine transform (MDCT) to obtain energy variances of each frequency point of a transformed audio frame, and determine whether the audio frame is the time-domain stationary signal according to the energy variances of each frequency point of the transformed audio frame.

24. The apparatus of claim 23, wherein, The determining module is specifically configured to determine that the audio frame is the time-domain stationary signal if a ratio of the energy of the filtered audio frame to the energy of the audio frame is greater than a second set threshold. The determining module is specifically configured to determine that the audio frame is not the time-domain stationary signal if the ratio of the energy of the filtered audio frame to the energy of the audio frame is less than the second set threshold.

25. The apparatus of claim 23, wherein, The determining module is specifically configured to determine that the audio frame is the time-domain stationary signal if the energy variances of each frequency point of the transformed audio frame are greater than a third set threshold, and determine that the audio frame is not the time-domain stationary signal if the energy variances of each frequency point of the transformed audio frame are less than the third set threshold.

26. The apparatus of any one of claims 21-25, wherein, The determining module is specifically configured to determine that the final encoding mode of the current audio frame is a frequency domain encoding mode when the initial encoding mode of the current audio frame and the final encoding mode of the previous audio frame are both frequency domain encoding modes; or determine that the final encoding mode of the current audio frame is a time domain encoding to frequency domain encoding switching encoding mode when the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a time domain encoding mode; or determine that the final encoding mode of the current audio frame is a frequency domain encoding mode when the initial encoding mode of the current audio frame is a frequency domain encoding mode and the final encoding mode of the previous audio frame is a time domain encoding to frequency domain encoding switching encoding mode; or determine that the final encoding mode of the current audio frame is a time domain encoding to frequency domain encoding switching encoding mode when the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain encoding to time domain encoding switching encoding mode; or determine that the final encoding mode of the current audio frame is a time domain encoding mode when the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain encoding to time domain encoding switching encoding mode; or determine that the final encoding mode of the current audio frame is a frequency domain encoding to time domain encoding switching encoding mode when the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain encoding to time domain encoding switching encoding mode; or determine that the final encoding mode of the current audio frame is a time domain encoding mode when the initial encoding mode of the current audio frame is a time domain encoding mode and the final encoding mode of the previous audio frame is a frequency domain encoding to time domain encoding switching encoding mode.

27. The apparatus of any of claims 21-26, wherein, When the final encoding mode is the time domain encoding mode, the encoding module is specifically configured to splice the last N / 2 sample data of the previous audio frame and the first N / 2 sample data of the current audio frame to obtain first data with a length of N, N representing the length of an audio frame; and perform time domain encoding on the first data.

28. The apparatus of any of claims 21-26, wherein, When the final encoding mode is the frequency domain encoding mode, the encoding module is specifically configured to splice the N sample data of the previous audio frame and the N sample data of the current audio frame to obtain second data with a length of 2N, N representing the length of an audio frame; perform windowing processing on the second data to obtain third data with a length of 2N; and perform frequency domain encoding on the third data.

29. The apparatus of any of claims 21-26, wherein, When the final encoding mode is the frequency domain encoding to the time domain encoding switching encoding mode, the encoding module is specifically configured to perform windowing processing on the N sample data of the previous audio frame to obtain fourth data with a length of N, N representing a length of an audio frame; perform odd symmetry folding processing on the fourth data with a center position as an odd symmetry point to obtain fifth data with a length of N / 2; splice the fifth data and the first N / 2 sample data of the current audio frame to obtain sixth data with a length of N; and perform time domain encoding on the sixth data.

30. The apparatus of any one of claims 21-26, wherein, When the final encoding mode is the time domain encoding to the frequency domain encoding switching encoding mode, and a total time delay of time domain encoding and decoding is N sample data, the encoding module is specifically configured to perform windowing processing on the N sample data of the current audio frame to obtain seventh data with a length of N, N representing a length of an audio frame; perform even symmetry folding processing on the seventh data with a center position as an even symmetry point to obtain eighth data with a length of N / 2; splice the last N / 2 sample data of the previous audio frame and the eighth data to obtain ninth data with a length of N; and perform time domain encoding on the ninth data.

31. The apparatus of any one of claims 21-26, wherein, When the final encoding mode is the time domain encoding to the frequency domain encoding switching encoding mode, and a total time delay of time domain encoding and decoding is N+D sample data, the encoding module is specifically configured to perform windowing processing on the N sample data of the current audio frame to obtain tenth data with a length of N; perform even symmetry folding processing on the tenth data with a center position as an even symmetry point to obtain eleventh data with a length of N / 2, N representing a length of an audio frame; perform even symmetry unfolding processing on the eleventh data with a center position as an even symmetry point to obtain twelfth data with a length of N, the first N / 2 sample data of the twelfth data being the eleventh data, and the last N / 2 sample data of the twelfth data being thirteenth data with a length of N / 2; splice the eleventh data and the first D sample data of the thirteenth data to obtain fourteenth data with a length of N / 2+D, D representing a time delay of QMF in the time domain encoding mode; perform interpolation processing on the fourteenth data to obtain fifteenth data with a length of N / 2; splice the last N / 2 sample data of the previous audio frame and the fifteenth data to obtain sixteenth data with a length of N; and perform time domain encoding on the sixteenth data.

32. The apparatus of any one of claims 21-31, wherein, The current audio frame is a pulse code modulation (PCM) signal, or the current audio frame is a signal obtained by performing quadrature mirror filtering (QMF) on the PCM signal, or the current audio frame is a residual signal obtained by performing inverse LPC filtering on the PCM signal.

33. An audio decoding apparatus, characterized by comprising: The method comprises: an acquisition module configured to acquire a bitstream; The decoding module is configured to parse the code stream to obtain code stream data to be decoded and indication information of a decoding mode, the decoding mode comprising a time domain decoding mode, a frequency domain decoding mode, a time domain decoding to frequency domain decoding switching decoding mode, or a frequency domain decoding to time domain decoding switching decoding mode; and decode the code stream data according to the decoding mode; the decoding mode is determined according to an initial encoding mode of a current audio frame and a final encoding mode of a previous audio frame; the initial encoding mode is determined according to a signal type of the current audio frame and a use scenario of the current audio frame, and the initial encoding mode comprises a time domain encoding mode or a frequency domain encoding mode, and the use scenario comprises a low-delay scenario or a high-audio-quality scenario.

34. The apparatus of claim 33, wherein, When the decoding mode is the time domain decoding mode, the decoding module is specifically configured to perform time domain decoding on the code stream data to obtain seventeenth data with a length of N, N representing the length of an audio frame; the last N / 2 sample points of the seventeenth data are the first N / 2 sample points of the current audio frame, and the first N / 2 sample points of the seventeenth data are the last N / 2 sample points of the previous audio frame of the current audio frame; the first N / 2 sample points of the previous audio frame are spliced with the first N / 2 sample points of the seventeenth data to obtain the data of the previous audio frame; the last N / 2 sample points of the seventeenth data are stored in the cache.

35. The apparatus of claim 33, wherein, When the decoding mode is the frequency domain decoding mode, the total time delay of time domain encoding and decoding is N+D sample points, and the decoding mode of the previous audio frame is the time domain decoding to frequency domain decoding switching decoding mode, the decoding module is specifically configured to perform frequency domain decoding on the code stream data to obtain eighteenth data with a length of 2N, N representing the length of an audio frame; the first N / 2 sample points of the first N sample points of the eighteenth data are spliced with D sample points after the first N / 2 sample points to obtain nineteenth data with a length of N / 2+D, D representing the time delay of a quadrature mirror filter QMF in the time domain encoding mode; the nineteenth data is subjected to interpolation processing to obtain twentieth data with a length of N / 2; the last N / 2 sample points of the first N sample points of the eighteenth data are spliced with D sample points before the last N / 2 sample points to obtain twenty-first data with a length of N / 2+D; the twenty-first data is subjected to interpolation processing to obtain twenty-second data with a length of N / 2; the twentieth data and the twenty-second data are spliced to obtain twenty-third data with a length of N; the twenty-third data and N sample points in the cache obtained through previous frequency domain decoding are subjected to windowing overlap processing to obtain the data of the previous audio frame; the last N sample points of the eighteenth data are stored in the cache.

36. The apparatus of claim 33, wherein, When the decoding mode is the frequency domain decoding mode, and a total time delay of time domain coding and decoding is N+D sample data, N represents a length of an audio frame, D represents a time delay of QMF in the time domain coding mode, and a decoding mode of a previous audio frame is the frequency domain decoding mode; or when the decoding mode is the frequency domain decoding mode, and the total time delay of time domain coding and decoding is N sample data, the decoding module is specifically configured to perform frequency domain decoding on the code stream data to obtain twenty-fourth data with a length of 2N; perform windowing overlap processing on a first N sample data of the twenty-fourth data and a last N sample data of twenty-fifth data with a length of 2N obtained by previous frequency domain decoding to obtain data of a previous audio frame; store the last N sample data of the twenty-fourth data in a buffer.

37. The apparatus of claim 33, wherein, When the decoding mode is switched from the frequency domain decoding mode to the time domain decoding mode, and the total time delay of time domain coding and decoding is N sample data, N represents the length of the audio frame, the decoding module is specifically configured to perform time domain decoding on the code stream data to obtain twenty-sixth data with a length of N; perform odd symmetry expansion processing on a first N / 2 sample data of the twenty-sixth data with a center position as an odd symmetry point to obtain twenty-seventh data with a length of N; perform windowing overlap processing on the twenty-seventh data and a last N sample data of twenty-eighth data with a length of 2N obtained by previous frequency domain decoding to obtain data of the previous audio frame with a length of N; store a last N / 2 sample data of the twenty-sixth data in the buffer.

38. The apparatus of claim 33, wherein, When the decoding mode is switched from the frequency domain decoding mode to the time domain decoding mode, and the total time delay of time domain coding and decoding is N+D sample data, N represents the length of the audio frame, and D represents the time delay of QMF in the time domain coding mode, the decoding module is specifically configured to perform time domain decoding on the code stream data to obtain twenty-ninth data with a length of N; perform interpolation processing on a last N / 2-D sample data of a first N / 2 sample data of the twenty-ninth data to obtain thirtieth data with a length of N / 2; perform odd symmetry expansion processing on the thirtieth data with a center position as an odd symmetry point to obtain thirty-first data with a length of N; perform interpolation processing on a last N / 2-D sample data of a first N / 2 sample data of a last N sample data of thirty-second data with a length of 2N obtained by previous frequency domain decoding to obtain thirty-third data with a length of N / 2; perform interpolation processing on a last N / 2-D sample data of a last N sample data of the thirty-second data to obtain thirty-fourth data with a length of N / 2; splice the thirty-third data and the thirty-fourth data to obtain thirty-fifth data with a length of N; perform windowing overlap processing on the thirty-first data and the thirty-fifth data to obtain data of the previous audio frame with a length of N; store a last N / 2 sample data of the twenty-ninth data in the buffer.

39. The apparatus of claim 33, wherein, When the decoding mode is switched to the frequency domain decoding mode from the time domain decoding mode, the decoding module is specifically configured to perform time domain decoding on the code stream data to obtain thirty-sixth data with a length of N, N representing a length of an audio frame; splice last N / 2 sample data of thirty-seventh data with a length of N obtained by last time domain decoding and first N / 2 sample data of the thirty-sixth data to obtain data of a last audio frame with a length of N; perform even symmetry expansion processing on the last N / 2 sample data of the thirty-sixth data with a center position as an even symmetry point to obtain thirty-eighth data with a length of N; and store the thirty-eighth data in a cache.

40. The apparatus of any one of claims 33-39, wherein, The last audio frame is a pulse code modulation (PCM) signal; or the last audio frame is a PCM signal obtained by quadrature mirror filtering (QMF); or the last audio frame is a PCM signal obtained by linear predictive coding (LPC) filtering.

41. An audio encoding device, characterized by The method comprises: one or more processors; a memory configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-12.

42. An audio decoding device, characterized by The method comprises: one or more processors; a memory configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 13-20.

43. A computer-readable storage medium, characterized in that, The computer program is executed on a computer to cause the computer to perform the method according to any one of claims 1-20.

44. A computer-readable storage medium, comprising: The encoded code stream is obtained according to the audio encoding method according to any one of claims 1-12.

45. A computer-readable storage medium, comprising: The audio frame is obtained according to the audio decoding method according to any one of claims 13-20.

Images