Encoding method, encoding apparatus, decoding method, decoding apparatus, and transmission system

Abstract

The present disclosure relates to an encoding method, an encoding apparatus, a decoding method, a decoding apparatus, and a transmission system, and relates to the technical field of audio processing. The encoding method comprises: encoding current frame audio information to generate main frame data; generating historical frame data, wherein the historical frame data comprises historical frame audio information, the encoding bit rate of the historical frame data being lower than the encoding bit rate of the main frame data; and generating a current frame bitstream on the basis of the main frame data and the historical frame data. The technical solution in the present disclosure can improve the anti-packet loss capability, thereby improving an audio transmission effect.

Description

Coding method, coding device, decoding method, decoding device and transmission system

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority to an application filed in China with application number 202311861467.0 and filing date December 29, 2023. The disclosed content of the Chinese application is hereby introduced as a whole into this application. Technical Field

[0003] The present disclosure relates to the field of audio processing technology, and in particular to an encoding method, an encoding device, a decoding method, a decoding device, a transmission system, and a computer-readable storage medium. Background Art

[0004] Interactive entertainment applications often feature music, such as live broadcasts, competitions, or KTV (Karaoke TV) waiting. In these scenarios, users generally have high expectations for audio quality. Even in weak network conditions, users expect a smooth music experience. This requires audio coding technology with high robustness against packet loss.

[0005] In the related art, in an RTC (Real-Time Communication) music scenario, a basic PLC (Packet Loss Concealment) algorithm in the Opus music coding mode is used as an anti-packet loss technology. Summary of the Invention

[0006] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: weak anti-packet loss capability, resulting in poor audio transmission effect.

[0007] In view of this, the present disclosure proposes a coding technology solution that can improve the anti-packet loss capability and thus improve the audio transmission effect.

[0008] According to some embodiments of the present disclosure, an encoding method is provided, including: encoding current frame audio information to generate main frame data; generating historical frame data, the historical frame data including historical frame audio information, the encoding bit rate of the historical frame data being lower than the encoding bit rate of the main frame data; and generating a current frame code stream based on the main frame data and the historical frame data.

[0009] In some embodiments, the historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than the frequency of the low-frequency band information; the number of bits of the spectral shape encoding of the high-frequency band information in the historical frame data of the current frame code stream is less than the number of bits of the spectral shape encoding of the high-frequency band information in the main frame data of the historical frame code stream.

[0010] In some embodiments, the number of bits of spectrum shape encoding of low-frequency band information in historical frame data of the current frame code stream is less than or equal to the number of bits of spectrum shape encoding of low-frequency band information in main frame data of the historical frame code stream.

[0011] In some embodiments, the historical frame data disables spectral shape encoding of high-band information.

[0012] In some embodiments, when the remaining bits of the historical frame data are lower than a threshold, the historical frame data disables spectrum shape coding of the high-frequency band information, and the remaining bits are the difference between available coding bits and used coding bits.

[0013] In some embodiments, the historical frame data includes skip information for indicating disabling of spectrum shape encoding of the high-frequency band information.

[0014] In some embodiments, the historical frame data includes spectrum energy information of the historical frame audio information and spectrum shape information of the low-frequency band information.

[0015] In some embodiments, the historical frame data includes a code stream header byte for indicating time offset information of the historical frame audio information relative to the current frame audio information.

[0016] In some embodiments, at least a portion of the high frequency band information of the historical frame audio information in the historical frame data is encoded using a bandwidth extension (BWE) algorithm.

[0017] In some embodiments, the historical frame data includes BWE code stream information for encoding the spectrum shape of the high frequency band information.

[0018] In some embodiments, the BWE code stream information is used to encode at least one of spectrum energy of the high-band information or bit allocation information of the high-band information.

[0019] In some embodiments, the historical frame data includes BWE status information, which is used to indicate whether BWE code stream information is written into the historical frame data.

[0020] In some embodiments, the historical frame data is encoded using a Constrained Energy Lapse Transform (CELT).

[0021] In some embodiments, the historical frame data includes an indication flag for indicating whether to perform spectrum shape encoding on the historical frame audio information.

[0022] According to other embodiments of the present disclosure, a decoding method is provided, including: receiving a current frame code stream, generating historical frame data of the current frame code stream by encoding historical frame audio information, and generating main frame data of the current frame code stream by encoding current frame audio information; in the event that packet loss occurs in the historical frame audio information, decoding the historical frame data to restore the historical frame audio information.

[0023] In some embodiments, the historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than the frequency of the low-frequency band information; the number of bits of the spectral shape encoding of the high-frequency band information in the historical frame data of the current frame code stream is less than the number of bits of the spectral shape encoding of the high-frequency band information in the main frame data of the historical frame code stream.

[0024] In some embodiments, the number of bits of spectrum shape encoding of low-frequency band information in historical frame data of the current frame code stream is less than or equal to the number of bits of spectrum shape encoding of low-frequency band information in main frame data of the historical frame code stream.

[0025] In some embodiments, the historical frame data disables spectral shape encoding of high-band information.

[0026] In some embodiments, when the remaining bits of the historical frame data are lower than a threshold, the historical frame data disables spectrum shape coding of the high-frequency band information, and the remaining bits are the difference between available coding bits and used coding bits.

[0027] In some embodiments, the historical frame data includes skip information for indicating disabling of spectrum shape encoding of the high-frequency band information.

[0028] In some embodiments, the historical frame data includes spectrum energy information of the historical frame audio information and spectrum shape information of the low-frequency band information.

[0029] In some embodiments, the historical frame data includes a code stream header byte for indicating time offset information of the historical frame audio information relative to the current frame audio information.

[0030] In some embodiments, at least a portion of the high frequency band information of the historical frame audio information in the historical frame data is encoded using a bandwidth extension (BWE) algorithm.

[0031] In some embodiments, the historical frame data includes BWE code stream information for encoding the spectrum shape of the high frequency band information.

[0032] In some embodiments, the BWE code stream information is used to encode at least one of spectrum energy of the high-band information or bit allocation information of the high-band information.

[0033] In some embodiments, the historical frame data includes BWE status information, which is used to indicate whether BWE code stream information is written into the historical frame data.

[0034] In some embodiments, the historical frame data is encoded using a Constrained Energy Lapse Transform (CELT).

[0035] In some embodiments, the historical frame data includes an indication flag for indicating whether to perform spectrum shape encoding on the historical frame audio information.

[0036] According to some further embodiments of the present disclosure, an encoding device is provided, including: a first encoding unit, used to encode current frame audio information to generate main frame data; a second encoding unit, used to generate historical frame data, the historical frame data including historical frame audio information, the encoding bit rate of the historical frame data being lower than the encoding bit rate of the main frame data; and a generation unit, used to generate a current frame code stream based on the main frame data and the historical frame data.

[0037] According to some further embodiments of the present disclosure, a decoding device is provided, including: a receiving unit, configured to receive a current frame code stream, wherein historical frame data of the current frame code stream is generated by encoding historical frame audio information, and main frame data of the current frame code stream is generated by encoding current frame audio information; and a decoding unit, configured to decode the historical frame data to restore the historical frame audio information in the event that packet loss occurs in the historical frame audio information.

[0038] According to some further embodiments of the present disclosure, a transmission system is provided, comprising: an encoding device for executing the encoding method in any one of the above embodiments; and a decoding device for executing the decoding method in any one of the above embodiments.

[0039] According to some further embodiments of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the encoding method or decoding method in any one of the above embodiments based on instructions stored in the memory device.

[0040] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the encoding method or decoding method in any of the above embodiments is implemented.

[0041] In the above embodiment, historical frame data is added to the current frame stream so that the current frame stream can carry the historical frame audio information. This can effectively combat the packet loss problem of the historical frame audio information, thereby improving the anti-packet loss capability and thus improving the audio transmission effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings:

[0044] FIG1 shows a flowchart of some embodiments of the encoding method of the present disclosure;

[0045] 2a to 2c are schematic diagrams illustrating some embodiments of the audio data packet disclosed herein;

[0046] FIG3 a shows a schematic diagram of some embodiments of the encoding method of the present disclosure;

[0047] FIG3 b shows a schematic diagram of some embodiments of the BWE encoding method of the present disclosure;

[0048] FIG4 shows a flowchart of some embodiments of the decoding method of the present disclosure;

[0049] FIG5 shows a block diagram of some embodiments of the encoding device of the present disclosure;

[0050] FIG6 shows a block diagram of some embodiments of a decoding device of the present disclosure;

[0051] FIG7 shows a block diagram of some embodiments of an electronic device of the present disclosure;

[0052] FIG8 shows a block diagram of another embodiment of the electronic device of the present disclosure;

[0053] FIG9 shows a block diagram of some embodiments of the transmission system of the present disclosure. DETAILED DESCRIPTION

[0054] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0055] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0056] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0057] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0058] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0059] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0060] As previously mentioned, addressing these technical issues requires improving the encoder's packet loss resistance. This disclosure enhances the encoder's packet loss resistance by implementing in-band FEC (Frame Error Concealment) technology in CELT-based audio encoding. This allows the characteristics of music signals to be exploited to efficiently encode music redundancy, achieving superior packet loss resistance while maintaining the same bandwidth.

[0061] For example, the technical solutions of the present disclosure can be implemented through the following embodiments.

[0062] FIG1 shows a flowchart of some embodiments of the encoding method of the present disclosure.

[0063] As shown in Figure 1, in step 110, the current frame audio information is encoded to generate main frame data. For example, the current frame audio information can be encoded using the CELT coding framework to generate main frame data.

[0064] For example, CELT coding utilizes psychoacoustic principles to divide the spectrum of the audio to be encoded into multiple subbands (e.g., 21). The energy and normalized subband spectral shape of each subband are quantized. The energy of each subband can be encoded first, followed by the shape of each subband (e.g., using the PVQ pyramid vector quantization coding algorithm). When encoding the subband shape, the available coding bits (bit-alloc) are preferentially allocated to the low-frequency band.

[0065] In some embodiments, an in-band FEC algorithm can be implemented in CELT audio coding to improve the anti-packet loss capability of CELT audio coding. For example, the above technical solution can be implemented through the relevant embodiments of steps 120 and 130.

[0066] In step 120, historical frame data is generated. The historical frame data includes historical frame audio information. The encoding bit rate of the historical frame data is lower than the encoding bit rate of the main frame data. For example, the encoding bit rate of the historical frame data (which may be called an intra-band frame) may not exceed a preset threshold (e.g., 24 kbps).

[0067] In some embodiments, the main frame data and the historical frame data use the same coding framework. For example, the same coding framework includes the CELT coding framework. For example, the CELT main frame data is encoded using a high bit rate, while the FEC historical frame data is encoded using a low bit rate and low complexity for higher quality.

[0068] For example, FEC historical frame data and main frame data use the same algorithm framework and can adopt the encoding logic in Table 1.

[0069] Table 1

[0070] In step 130, a current frame code stream is generated based on the main frame data and the historical frame data. For example, the current frame code stream can be generated by the embodiments in Figures 2a to 2c to implement the above technical solution.

[0071] 2a to 2c are schematic diagrams showing some embodiments of the audio data packet of the present disclosure.

[0072] As shown in Figure 2a, in the current frame (such as the nth frame) data packet using CELT for audio encoding, in addition to carrying the audio information of the current frame, it also carries the audio information of the historical frame (such as the nkth frame) to implement the in-band FEC algorithm in the CELT audio encoding.

[0073] For example, the encoding bitrate of historical frame data is lower than that of primary frame data. Implementing an in-band FEC algorithm in CELT audio coding can meet the following requirements: low-bitrate historical frame data, such as an FEC encoding bitrate of no more than 24kbps; low-complexity coding, such as ensuring that the increase in coding complexity after adding historical frame data does not exceed a preset threshold, such as 25%; and high-quality audio, such as ensuring that the MOS (Mean Opinion Score) improvement exceeds a preset threshold, such as 1.0, in packet loss scenarios.

[0074] In some embodiments, the historical frame data includes a code stream header byte for indicating time offset information of the historical frame audio information relative to the current frame audio information.

[0075] As shown in Figure 2b, the current frame codestream can include a toc field, a CELT payload format field, and an extended payload field. The toc field (part of the codestream header byte) and the CELT payload format field constitute the main frame data, which is used to store the encoded result of the current frame's audio information.

[0076] The extended payload field includes the toh field and the (nk)th frame CELT FEC payload field, which are used to store the encoded audio information of the historical frame. The (nk)th frame CELT FEC payload field is the in-band FEC field of the (nk)th frame CELT. The toh field (e.g., which can occupy 1 byte) can be used to store information about the offset between the historical frame and the current frame, such as the kth frame; the (nk)th frame CELT FEC payload field can be used to store the encoded audio information of the (nk)th frame.

[0077] In some embodiments, the historical frame data is used to restore the historical frame audio information when packet loss occurs in the historical frame audio information.

[0078] For example, when the decoding end receives the nth frame data packet, in addition to obtaining the audio information of the nth frame through decoding, the audio information of the nkth frame can also be obtained through decoding.

[0079] In this way, when the audio signal of the nkth frame is lost, the historical frame data of the nth frame data packet can be used to restore the audio signal of the nkth frame, so as to improve the packet loss resistance during the audio transmission process, thereby improving the reliability of audio transmission.

[0080] As shown in Figure 2c, the four packets in the first row are from frames n to n+3, each containing historical frame data used to store audio information from a historical frame offset by three frames from the current frame. During data transmission to the decoder, the packet for frame n was lost, while the remaining four frames were not.

[0081] After the decoder side finishes playing the audio information of the n-1th frame, when it needs to decode the data packet of the nth frame (the data packets of the n+1th, n+2th and n+3th frames are to be decoded), it is detected that the data packet of the nth frame has been lost; in the above case, the historical frame data of the received data packet of the n+3th frame can be decoded to restore the audio information of the nth frame.

[0082] In the above embodiment, by encoding historical frame audio information to generate historical frame data, high-quality audio recovery can be achieved in the event of packet loss, thereby enhancing transmission reliability in audio scenarios such as RTC.

[0083] In some embodiments, the historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than the frequency of the low-frequency band information. For example, the frequency of the high-frequency band information is higher than a frequency threshold, and the frequency of the low-frequency band information is lower than the frequency threshold, such as the frequency threshold may be 8 kHz.

[0084] In some embodiments, the historical frame data disables the encoding of the spectrum shape of the high-frequency band information. For example, the spectrum shape information of the high-frequency band information is not encoded in the historical frame data.

[0085] In some embodiments, when the remaining bits of the historical frame data are lower than a threshold, the historical frame data disables spectrum shape coding of the high-frequency band information, and the remaining bits are the difference between available coding bits and used coding bits.

[0086] For example, the remaining bits are the bits remaining in the historical frame data after the spectral energy encoding of the historical frame audio information and the spectral shape encoding of the low-frequency band information are performed. For example, the historical frame data includes the spectral energy information of the historical frame audio information and the spectral shape information of the low-frequency band information.

[0087] For example, FEC historical frame data uses a low-bitrate encoding scheme and the CELT coding framework. Prioritizing the energy coding quality of each subband and the spectral shape coding quality of the low-frequency band, this reduces the coding bitrate and improves the coding performance. For example, the PVQ (Pyramid Vector Quantization) bit allocation logic for FEC historical frame data can be modified to disable PVQ bit allocation for high-frequency band information.

[0088] In this way, the PVQ encoding bit rate can be reduced, thereby improving the encoding effect.

[0089] In some embodiments, the remaining bits are used to perform at least one of normalized spectrum shape vector quantization, band energy refinement, spectrum collapse mitigation, or residual energy quantization on the historical frame data. For example, the band energy refinement and residual energy quantization of the FEC historical frame data can be adapted.

[0090] For example, the remaining bits of FEC historical frame data are mainly used for normalized spectrum in-band PVQ of FEC historical frame data, fine energy quantization of historical frame data, anti-spectral collapse of historical frame data, and residual energy quantization processing of historical frame data.

[0091] For example, the PVQ of the historical frame data prioritizes bit allocation in the low frequency band; if the bit allocation in the high frequency band is insufficient, the PVQ bit allocation in the high frequency band is disabled.

[0092] In some embodiments, the historical frame data includes skip information for indicating disabling of spectrum shape coding of high frequency band information. For example, for a high frequency band with insufficient bit allocation, PVQ of the high frequency band is skipped by encoding SKIP information.

[0093] In some embodiments, the available number of bits of the historical frame data is determined according to the encoding rate of the historical frame data and the number of bits required for the spectrum energy information of the low-frequency band information, and the available number of bits of the historical frame data is less than the available number of bits of the main frame data.

[0094] For example, if the main frame data is encoded using a high bit rate, the number of available bits (nbCompressedBytes) is relatively large. However, if the historical frame data is encoded using a low bit rate and uses the in-band FEC rate control logic, the calculated number of available bits for the FEC historical frame data (nbCompressedBytes_fec) is relatively small. The calculation of the available bits for the historical frame data requires considering the encoding bit rate of the FEC historical frame data and the number of PVQ bits required for CELT low-band encoding.

[0095] In some embodiments, the spectrum shape of the high-frequency band information of the historical frame audio information is restored by spectrum folding or noise filling.

[0096] For example, when the historical frame data skips the spectral shape encoding of the high-frequency band of the historical frame audio information through the SKIP information, if the decoding side wants to restore the spectral shape of the high-frequency band of the historical frame audio information, the spectral shape of the high-frequency band can be restored by spectral folding or noise filling.

[0097] FIG3 a shows a schematic diagram of some embodiments of the encoding method of the present disclosure.

[0098] As shown in Figure 3a, under the CELT coding framework, after stereo analysis processing, historical frame data is generated through steps such as historical frame data bit rate calculation, historical frame data bit allocation, historical frame data energy quantization, normalized spectrum in-band PVQ, and historical frame data anti-spectral collapse and energy residual quantization.

[0099] During the historical frame data bitrate calculation process, the main frame data uses a high bitrate encoding method, and the number of available bits (nbCompressedBytes) is relatively large. However, the historical frame data uses a low bitrate encoding method and uses the in-band FEC bitrate control logic, so the calculated number of available bits for FEC historical frame data (nbCompressedBytes_fec) is relatively small. The calculation of the available bits for historical frame data requires considering the encoding bitrate of the FEC historical frame data and the number of PVQ bits required for CELT low-band encoding.

[0100] During the bit allocation process for historical frame data, the remaining bits of the FEC historical frame data are primarily used for normalized in-band spectral quality (PVQ) of the FEC historical frame data, fine-grained energy quantization of the historical frame data, countermeasures against spectrum collapse, and residual energy quantization of the historical frame data. PVQ of historical frame data prioritizes bit allocation in the low-frequency band; if the bit allocation for the high-frequency band is insufficient, PVQ bit allocation in the high-frequency band is disabled.

[0101] In the process of normalizing the intra-band PVQ of the spectrum, for high-frequency bands with insufficient bit allocation, PVQ of the high-frequency bands is skipped by encoding Skip information.

[0102] In some embodiments, each symbol in the historical frame data may be configured according to the embodiment in Table 2.

[0103] Table 2

[0104] In some embodiments, at least a portion of the high-frequency band information of the historical frame audio information in the historical frame data is encoded using a bandwidth extension (BWE) algorithm. For example, the BWE code stream information is used to encode the spectrum shape of the high-frequency band information.

[0105] In some embodiments, the BWE code stream information is used to encode at least one of spectrum energy of the high-band information or bit allocation information of the high-band information.

[0106] For example, when encoding high-bitrate CELT main frame data, a low-bitrate, low-complexity, high-quality encoding method can be used to generate an FEC in-band code stream; the FEC in-band code stream contains a BWE code stream, which is used to recover audio information in the high-frequency band (such as 8-20kHz).

[0107] In some embodiments, the historical frame data is encoded using CELT. For example, high-frequency band information encoded using CELT in the historical frame data is deleted; the high-frequency band information is encoded using the BWE algorithm to generate BWE code stream information. For example, the generated BWE code stream information can replace the CELT-encoded high-frequency band information.

[0108] For example, when BWE code stream information is introduced into FEC historical frame data, coding information of the high frequency band in the FEC intra-band code frame of the original CELT coding frame may be removed.

[0109] For example, CELT coding is used for audio information in the low frequency band (such as 0-8kHz), and BWE code stream information is inserted at the appropriate position of the FEC historical frame data, and the coding information of the high frequency band in the CELT code stream is removed.

[0110] For example, for high-frequency audio information (such as 8-20kHz), BWE encoding is used. The encoding method of FEC historical frame data uses information such as the energy of the high-frequency band, the spectrum shape of the high-frequency band, and the bit allocation of the high-frequency band in the BWE algorithm.

[0111] In the above embodiment, a more efficient BWE technology is introduced, and BWE code stream information is added to the FEC historical frame data, which improves the recovery quality of the FEC historical frame data for high-frequency band audio information, thereby improving the transmission reliability and processing effect of the audio information.

[0112] In some embodiments, the introduction of BWE technology can be achieved through the embodiment in FIG. 3 b .

[0113] FIG3 b shows a schematic diagram of some embodiments of the BWE encoding method of the present disclosure.

[0114] As shown in Figure 3b, under the CELT coding framework, after time-frequency analysis, historical frame data is generated through the following steps: historical frame data copying and BWE encoding, historical frame data energy coarse quantization, in-band time-frequency coding, in-band stereo analysis, historical frame data bit rate calculation, historical frame data bit allocation, historical frame data energy fine quantization, normalized spectrum in-band PVQ, and historical frame data anti-spectral collapse and energy residual quantization.

[0115] During the BWE encoding process, the historical frame data may include BWE state information, which indicates whether BWE bitstream information is written to the historical frame data. For example, the BWE_state field (used to indicate whether the BWE state is on) and BWE bitstream information (such as the energy of the high-band, the spectrum shape of the high-band, and the bit allocation of the high-band) are encoded.

[0116] For example, when BWE_state=0, the BWE state is disabled and the encoding method in Figure 3a can be used; when BWE_state=1, the BWE state is enabled, the BWE code stream information is written into the historical frame data, and the code stream information of the high frequency band in the CELT frame FEC in-band coding is removed.

[0117] In some embodiments, each symbol in the historical frame data may be configured according to the embodiment in Table 3 to implement the introduction of the BWE technology.

[0118] Table 3

[0119] In this way, the decoder can use the CELT framework's bitstream information to recover low-frequency audio information and the BWE bitstream information to recover high-frequency audio information. BWE enables guided high-frequency spectrum recovery, which improves audio recovery compared to techniques such as spectral folding, thereby enhancing audio transmission reliability and audio processing performance.

[0120] FIG4 shows a flowchart of some embodiments of the decoding method of the present disclosure.

[0121] As shown in FIG4 , in step 410 , a current frame code stream is received, historical frame data of the current frame code stream is generated by encoding historical frame audio information, and main frame data of the current frame code stream is generated by encoding current frame audio information.

[0122] In step 420 , when packet loss occurs in the historical frame audio information, the historical frame data is decoded to restore the historical frame audio information.

[0123] In some embodiments, the historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than the frequency of the low-frequency band information; the number of bits of the spectral shape encoding of the high-frequency band information in the historical frame data of the current frame code stream is less than the number of bits of the spectral shape encoding of the high-frequency band information in the main frame data of the historical frame code stream.

[0124] In some embodiments, the number of bits of spectrum shape encoding of low-frequency band information in historical frame data of the current frame code stream is less than or equal to the number of bits of spectrum shape encoding of low-frequency band information in main frame data of the historical frame code stream.

[0125] In some embodiments, the historical frame data disables spectral shape encoding of high-band information.

[0126] In some embodiments, when the remaining bits of the historical frame data are lower than a threshold, the historical frame data disables spectrum shape coding of the high-frequency band information, and the remaining bits are the difference between available coding bits and used coding bits.

[0127] In some embodiments, the historical frame data includes skip information for indicating disabling of spectrum shape encoding of the high-frequency band information.

[0128] In some embodiments, the historical frame data includes spectrum energy information of the historical frame audio information and spectrum shape information of the low-frequency band information.

[0129] In some embodiments, the historical frame data includes a code stream header byte for indicating time offset information of the historical frame audio information relative to the current frame audio information.

[0130] In some embodiments, at least a portion of the high frequency band information of the historical frame audio information in the historical frame data is encoded using a bandwidth extension (BWE) algorithm.

[0131] In some embodiments, the historical frame data includes BWE code stream information for encoding the spectrum shape of the high frequency band information.

[0132] In some embodiments, the BWE code stream information is used to encode at least one of spectrum energy of the high-band information or bit allocation information of the high-band information.

[0133] In some embodiments, the historical frame data includes BWE status information, which is used to indicate whether BWE code stream information is written into the historical frame data.

[0134] In some embodiments, the historical frame data is encoded using a Constrained Energy Lapse Transform (CELT).

[0135] In some embodiments, the historical frame data includes an indication flag for indicating whether to perform spectrum shape encoding on the historical frame audio information.

[0136] FIG5 shows a block diagram of some embodiments of the encoding apparatus of the present disclosure.

[0137] As shown in Figure 5, the encoding device 5 includes: a first encoding unit 51, which is used to encode the current frame audio information to generate main frame data; a second encoding unit 52, which is used to generate historical frame data, the historical frame data includes historical frame audio information, and the encoding bit rate of the historical frame data is lower than the encoding bit rate of the main frame data; a generation unit 53, which is used to generate a current frame code stream based on the main frame data and the historical frame data.

[0138] In some embodiments, the historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than the frequency of the low-frequency band information; the number of bits of the spectral shape encoding of the high-frequency band information in the historical frame data of the current frame code stream is less than the number of bits of the spectral shape encoding of the high-frequency band information in the main frame data of the historical frame code stream.

[0139] In some embodiments, the number of bits of spectrum shape encoding of low-frequency band information in historical frame data of the current frame code stream is less than or equal to the number of bits of spectrum shape encoding of low-frequency band information in main frame data of the historical frame code stream.

[0140] In some embodiments, the historical frame data disables spectral shape encoding of high-band information.

[0141] In some embodiments, when the remaining bits of the historical frame data are lower than a threshold, the historical frame data disables spectrum shape coding of the high-frequency band information, and the remaining bits are the difference between available coding bits and used coding bits.

[0142] In some embodiments, the historical frame data includes skip information for indicating disabling of spectrum shape encoding of the high-frequency band information.

[0143] In some embodiments, the historical frame data includes spectrum energy information of the historical frame audio information and spectrum shape information of the low-frequency band information.

[0144] In some embodiments, the historical frame data includes a code stream header byte for indicating time offset information of the historical frame audio information relative to the current frame audio information.

[0145] In some embodiments, at least a portion of the high frequency band information of the historical frame audio information in the historical frame data is encoded using a bandwidth extension (BWE) algorithm.

[0146] In some embodiments, the historical frame data includes BWE code stream information for encoding the spectrum shape of the high frequency band information.

[0147] In some embodiments, the BWE code stream information is used to encode at least one of spectrum energy of the high-frequency band information or bit allocation information of the high-frequency band information.

[0148] In some embodiments, the historical frame data includes BWE status information, which is used to indicate whether BWE code stream information is written into the historical frame data.

[0149] In some embodiments, the historical frame data is encoded using a Constrained Energy Lapse Transform (CELT).

[0150] In some embodiments, the historical frame data includes an indication flag for indicating whether to perform spectrum shape encoding on the historical frame audio information.

[0151] FIG6 shows a block diagram of some embodiments of a decoding device of the present disclosure.

[0152] As shown in Figure 6, the decoding device 6 includes: a receiving unit 61, which is used to receive the current frame code stream, the historical frame data of the current frame code stream is generated by encoding the historical frame audio information, and the main frame data of the current frame code stream is generated by encoding the current frame audio information; a decoding unit 62, which is used to decode the historical frame data in the event that the historical frame audio information is lost to restore the historical frame audio information.

[0153] In some embodiments, the historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than the frequency of the low-frequency band information; the number of bits of the spectral shape encoding of the high-frequency band information in the historical frame data of the current frame code stream is less than the number of bits of the spectral shape encoding of the high-frequency band information in the main frame data of the historical frame code stream.

[0154] In some embodiments, the number of bits of spectrum shape encoding of low-frequency band information in historical frame data of the current frame code stream is less than or equal to the number of bits of spectrum shape encoding of low-frequency band information in main frame data of the historical frame code stream.

[0155] In some embodiments, the historical frame data disables spectral shape encoding of high-band information.

[0156] In some embodiments, when the remaining bits of the historical frame data are lower than a threshold, the historical frame data disables spectrum shape coding of the high-frequency band information, and the remaining bits are the difference between available coding bits and used coding bits.

[0157] In some embodiments, the historical frame data includes skip information for indicating disabling of spectrum shape encoding of the high-frequency band information.

[0158] In some embodiments, the historical frame data includes spectrum energy information of the historical frame audio information and spectrum shape information of the low-frequency band information.

[0159] In some embodiments, the historical frame data includes a code stream header byte for indicating time offset information of the historical frame audio information relative to the current frame audio information.

[0160] In some embodiments, at least a portion of the high frequency band information of the historical frame audio information in the historical frame data is encoded using a bandwidth extension (BWE) algorithm.

[0161] In some embodiments, the historical frame data includes BWE code stream information for encoding the spectrum shape of the high frequency band information.

[0162] In some embodiments, the BWE code stream information is used to encode at least one of spectrum energy of the high-band information or bit allocation information of the high-band information.

[0163] In some embodiments, the historical frame data includes BWE status information, which is used to indicate whether BWE code stream information is written into the historical frame data.

[0164] In some embodiments, the historical frame data is encoded using a Constrained Energy Lapse Transform (CELT).

[0165] In some embodiments, the historical frame data includes an indication flag for indicating whether to perform spectrum shape encoding on the historical frame audio information.

[0166] FIG7 shows a block diagram of some embodiments of the electronic device of the present disclosure.

[0167] As shown in Figure 7, the electronic device 7 of this embodiment includes: a memory 71 and a processor 72 coupled to the memory 71, and the processor 72 is configured to execute the encoding method or decoding method in any embodiment of the present disclosure based on the instructions stored in the memory 71.

[0168] The memory 71 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, application programs, a boot loader, a database, and other programs.

[0169] FIG8 shows a block diagram of other embodiments of the electronic device of the present disclosure.

[0170] As shown in FIG8 , the electronic device 8 of this embodiment includes a memory 810 and a processor 820 coupled to the memory 810 . The processor 820 is configured to execute the encoding method or decoding method in any one of the aforementioned embodiments based on instructions stored in the memory 810 .

[0171] The memory 810 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs.

[0172] The electronic device 8 may also include an input / output interface 830, a network interface 840, a storage interface 850, and the like. These interfaces 830, 840, 850, as well as the memory 810 and the processor 820, may be connected, for example, via a bus 860. The input / output interface 830 provides a connection interface for input / output devices such as a display, mouse, keyboard, touch screen, microphone, and speakers. The network interface 840 provides a connection interface for various networked devices. The storage interface 850 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0173] FIG9 shows a block diagram of some embodiments of the transmission system of the present disclosure.

[0174] As shown in FIG9 , the transmission system 9 includes: an encoding device 91 for executing the encoding method in any one of the above embodiments; and a decoding device 92 for executing the decoding method in any one of the above embodiments.

[0175] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media, including but not limited to magnetic disk storage, CD-ROMs, optical storage, and the like, containing computer-usable program code.

[0176] The encoding method, encoding device, decoding method, decoding device, transmission system, and computer-readable storage medium according to the present disclosure have been described in detail. To avoid obscuring the concept of the present disclosure, some details known in the art have been omitted. Based on the above description, those skilled in the art will fully understand how to implement the technical solutions disclosed herein.

[0177] The methods and systems of the present disclosure may be implemented in many ways. For example, the methods and systems of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Therefore, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0178] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that modifications may be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A coding method, comprising: Encoding the audio information of the current frame to generate main frame data; Generating historical frame data, where the historical frame data includes historical frame audio information, and the coding bit rate of the historical frame data is lower than that of the main frame data; Generating the current frame bitstream according to the main frame data and the historical frame data.

2. The coding method according to claim 1, wherein: The historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than that of the low-frequency band information; The number of bits for encoding the spectral shape of the high-frequency band information in the historical frame data of the current frame bitstream is less than the number of bits for encoding the spectral shape of the high-frequency band information in the main frame data of the historical frame bitstream.

3. The encoding method according to claim 2, wherein, The number of bits for encoding the spectral shape of the low-frequency band information in the historical frame data of the current frame bitstream is less than or equal to the number of bits for encoding the spectral shape of the low-frequency band information in the main frame data of the historical frame bitstream.

4. The encoding method according to claim 2, wherein, The historical frame data disables encoding of the spectral shape of the high-frequency band information.

5. The encoding method according to claim 4, wherein, In the case where the remaining bits of the historical frame data are lower than a threshold, the historical frame data disables encoding of the spectral shape of the high-frequency band information, and the remaining bits are the difference between the available coding bits and the used coding bits.

6. The encoding method according to claim 4, wherein, The historical frame data includes skip information for indicating disabling encoding of the spectral shape of the high-frequency band information.

7. The encoding method according to claim 2, wherein, The historical frame data includes the spectral energy information of the historical frame audio information and the spectral shape information of the low-frequency band information.

8. The encoding method according to any one of claims 1-7, wherein, The historical frame data includes a bitstream header byte for indicating the time offset information of the historical frame audio information relative to the current frame audio information.

9. The encoding method according to any one of claims 1-7, wherein, At least a part of the high-frequency band information of the historical frame audio information in the historical frame data is encoded using the bandwidth expansion BWE algorithm.

10. The encoding method according to claim 9, wherein, The historical frame data includes BWE bitstream information for encoding the spectral shape of the high-frequency band information.

11. The encoding method according to claim 10, wherein, The BWE bitstream information is used to encode at least one of the spectral energy of the high-frequency band information or the bit allocation information of the high-frequency band information.

12. The encoding method according to claim 10, wherein, The historical frame data includes BWE status information for indicating whether the BWE bitstream information is written in the historical frame data.

13. The encoding method according to claims 1-7, wherein, The historical frame data is encoded using the constrained energy overlap transform CELT.

14. The encoding method according to any one of claims 1-7, wherein, The historical frame data includes an indication identifier for indicating whether to encode the spectral shape of the historical frame audio information.

15. A decoding method, comprising: Receiving a current frame bitstream, where the historical frame data of the current frame bitstream is generated by encoding historical frame audio information, and the main frame data of the current frame bitstream is generated by encoding current frame audio information; In the case where the historical frame audio information is lost, decoding the historical frame data to recover the historical frame audio information.

16. The decoding method according to claim 15, wherein: The historical frame audio information includes high-frequency band information and low-frequency band information, and the frequency of the high-frequency band information is higher than that of the low-frequency band information; The number of bits for encoding the spectral shape of the high-band information in the historical frame data of the current frame stream is less than the number of bits for encoding the spectral shape of the high-band information in the main frame data of the historical frame stream.

17. The decoding method according to claim 16, wherein, The number of bits for encoding the spectral shape of the low-band information in the historical frame data of the current frame stream is less than or equal to the number of bits for encoding the spectral shape of the low-band information in the main frame data of the historical frame stream.

18. The decoding method according to claim 16, wherein, Encoding of the spectral shape of the high-band information is disabled in the historical frame data.

19. The decoding method according to claim 18, wherein, When the remaining bits in the historical frame data are lower than a threshold, encoding of the spectral shape of the high-band information is disabled in the historical frame data, where the remaining bits are the difference between the available encoding bits and the used encoding bits.

20. The decoding method according to claim 18, wherein, The historical frame data includes skip information for indicating that encoding of the spectral shape of the high-band information is disabled.

21. The decoding method according to claim 16, wherein, The historical frame data includes the spectral energy information of the historical frame audio information and the spectral shape information of the low-band information.

22. The decoding method according to any one of claims 15-21, wherein, The historical frame data includes a bitstream header byte for indicating the time offset information of the historical frame audio information relative to the current frame audio information.

23. The decoding method according to any one of claims 15-21, wherein, At least a part of the high-band information of the historical frame audio information in the historical frame data is encoded using the bandwidth expansion BWE algorithm.

24. The decoding method according to claim 23, wherein, The historical frame data includes BWE bitstream information for encoding the spectral shape of the high-band information.

25. The decoding method according to claim 24, wherein, The BWE bitstream information is used to encode at least one of the spectral energy of the high-band information or the bit allocation information of the high-band information.

26. The decoding method according to claim 24, wherein, The historical frame data includes BWE status information for indicating whether the BWE bitstream information is written in the historical frame data.

27. The decoding method according to claims 15-21, wherein, The historical frame data is encoded using the Constrained Energy Lapped Transform CELT.

28. The decoding method according to any one of claims 15-21, wherein, The historical frame data includes an indication flag for indicating whether spectral shape encoding is performed on the historical frame audio information.

29. An encoding device, comprising: A first encoding unit for encoding current frame audio information to generate main frame data; A second encoding unit for generating historical frame data, where the historical frame data includes historical frame audio information, and the encoding bit rate of the historical frame data is lower than the encoding bit rate of the main frame data; A generating unit for generating a current frame stream according to the main frame data and the historical frame data.

30. A decoding device, comprising: A receiving unit for receiving a current frame stream, where the historical frame data of the current frame stream is generated by encoding historical frame audio information, and the main frame data of the current frame stream is generated by encoding current frame audio information; A decoding unit for decoding the historical frame data to recover the historical frame audio information when packet loss occurs in the historical frame audio information.

31. A transmission system, comprising: An encoding device for performing the encoding method according to any one of claims 1-14; A decoding device for performing the decoding method according to any one of claims 15-28.

32. An electronic device, comprising: A memory; And A processor coupled to the memory, the processor being configured to execute the encoding method according to any one of claims 1-14, or the decoding method according to any one of claims 15-28, based on instructions stored in the memory.

33. A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the encoding method according to any one of claims 1-14, or the decoding method according to any one of claims 15-28.

Images