Audio encoding method, audio decoding method, and related apparatuses

By combining AI and non-AI modules with a hybrid encoder to segment and encode audio signals, the problem of sound quality degradation at low bit rates is solved, and the sound quality of audio encoding is improved while the computational complexity is controlled.

WO2026188926A9PCT designated stage Publication Date: 2026-11-12HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/143941
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-03-10
Filing Date
2025-12-19
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

At low bitrates, traditional audio encoding and decoding methods cannot balance the reduction in bitrate with the guarantee of sound quality, resulting in a decline in audio signal quality.

Method used

A hybrid encoder is used, combining an artificial intelligence (AI) encoding module and a non-AI encoding module, to segment and encode the audio signal. The powerful compression capability of the AI ​​encoding module is used to improve the sound quality, while the low complexity of the non-AI encoding module is used to control the computational complexity.

Benefits of technology

While controlling computational complexity, it improves the sound quality of audio encoding, making up for the problem of poor sound quality of traditional audio codecs at low bit rates.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025143941_12112026_PF_FP_ABST
    Figure CN2025143941_12112026_PF_FP_ABST
Patent Text Reader

Abstract

An audio encoding method, an audio decoding method, and related apparatuses, relating to the field of data compression. The audio encoding method comprises: determining a first encoder from among a plurality of encoders on the basis of a first bitrate; if the first encoder is a hybrid encoder, acquiring a plurality of sub-signals on the basis of a current audio signal frame, wherein the hybrid encoder comprises an AI encoding module and a non-AI encoding module; and using the AI encoding module to encode a first type of sub-signals among the plurality of sub-signals, and using the non-AI encoding module to encode a second type of sub-signals among the plurality of sub-signals. In other words, one portion of the sub-signals is encoded using the AI encoding module, and the other portion of the sub-signals is encoded using the non-AI encoding module, thereby leveraging the powerful compression capability of the AI encoding module to improve the quality of encoded audio, while leveraging the low complexity of the non-AI encoding module to control computational complexity. In this way, the quality of encoded audio is improved while controlling computational complexity, addressing the problem of poor audio quality in conventional audio codecs at low bitrates.
Need to check novelty before this filing date? Find Prior Art

Description

Audio encoding and decoding methods and related devices

[0001] This application claims priority to Chinese Patent Application No. 202510287063.8, filed on March 10, 2025, entitled "Audio Encoding and Decoding Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data compression, and in particular to an audio encoding / decoding method and related apparatus. Background Technology

[0003] As living standards improve, people's demand for high-quality audio continues to increase. To better utilize limited bandwidth for audio signal transmission, the encoding end typically encodes the audio signal into a bitstream and transmits the bitstream to the decoding end. The decoding end then reconstructs the audio signal based on the bitstream. However, in certain scenarios, the bitrate of the audio signal can drop sharply. In such cases, balancing the reduction in bitrate with ensuring the sound quality of the audio signal becomes a pressing problem to solve. Summary of the Invention

[0004] This application provides an audio encoding / decoding method and related apparatus, which can improve the sound quality of audio encoding while controlling computational complexity, thus overcoming the problem of poor sound quality in traditional audio encoding / decoding at low bit rates. The technical solution is as follows:

[0005] In a first aspect, an audio encoding method is provided, the method comprising: determining a first encoder from a plurality of encoders based on a first bitrate, the first bitrate being the encoding bitrate of a current frame audio signal, the first encoder being used to encode the current frame audio signal; if the first encoder is a hybrid encoder, acquiring a plurality of sub-signals based on the current frame audio signal, the hybrid encoder including an artificial intelligence (AI) encoding module and a non-AI encoding module; encoding a first type of sub-signal among the plurality of sub-signals using the AI ​​encoding module, and encoding a second type of sub-signal among the plurality of sub-signals using the non-AI encoding module to obtain first encoded data, the bitstream corresponding to the current frame audio signal including the first encoded data.

[0006] AI encoding modules possess powerful compression capabilities (for the same data, the encoding bitrate required by AI encoding modules is approximately one-third that of non-AI encoding modules; that is, the AI ​​encoding effect at a bitrate of 6kbps is equivalent to the non-AI encoding effect at a bitrate of around 15kbps). Using AI encoding modules to encode audio signals typically requires a lower bitrate, supporting bitrates below 8kbps, and even compressing narrowband audio signals to below 1kbps, but the compression complexity is usually high. Non-AI encoding modules generally have lower compression complexity, but at low bitrates, using them to encode audio signals degrades the audio quality. Therefore, combining AI and non-AI encoding modules—encoding a portion of the multiple sub-signals with AI and another portion with non-AI—leverages the powerful compression capabilities of AI to improve audio quality while controlling computational complexity with the lower complexity of non-AI modules. This approach improves audio quality while controlling computational complexity, overcoming the poor audio quality of traditional audio codecs at low bitrates.

[0007] In one possible implementation, each of the multiple encoders corresponds to a bitrate range, which is the bitrate range supported by the corresponding encoder. The encoder corresponding to the bitrate range with the first bitrate among the multiple encoders is determined as the first encoder. That is, multiple encoders are deployed at the encoding end, and different encoders correspond to different bitrate ranges. In this way, as the encoding bitrate changes dynamically, the corresponding encoder can be selected to encode the audio signal, thereby adapting to different scenarios and enabling continuous transmission of the audio signal.

[0008] As an example, the multiple encoders include an AI encoder, a hybrid encoder, and a non-AI encoder. The hybrid encoder includes both AI and non-AI encoding modules. The bitrate within the bitrate range corresponding to the AI ​​encoder is lower than the bitrate within the bitrate range corresponding to the hybrid encoder, and the bitrate within the bitrate range corresponding to the hybrid encoder is lower than the bitrate within the bitrate range corresponding to the non-AI encoder.

[0009] It should be understood that an AI encoder refers to an encoder that uses AI encoding technology, a hybrid encoder refers to an encoder that uses both AI and non-AI encoding technologies, and a non-AI encoder refers to an encoder that uses only non-AI encoding technologies. Non-AI encoding technology can be understood as traditional encoding technology. Furthermore, the AI ​​encoding technology used by an AI encoder may be the same as or different from the AI ​​encoding technology used by a hybrid encoder, and the non-AI encoding technology used by a non-AI encoder may be the same as or different from the non-AI encoding technology used by a hybrid encoder.

[0010] In one possible implementation, the bitrate range corresponding to the AI ​​encoder is less than or equal to the first reference bitrate. The bitrate range corresponding to the hybrid encoder is greater than the first reference bitrate and less than the second reference bitrate. The bitrate range corresponding to the non-AI encoder is greater than or equal to the second reference bitrate. The first reference bitrate is less than the second reference bitrate. For example, the first reference bitrate is 8kbps, 10kbps, 12kbps, 14kbps, or 16kbps, and the second reference bitrate is 32kbps.

[0011] In one possible implementation, the multiple sub-signals are multiple frequency band signals, which are obtained by dividing the spectrum of the current frame audio signal after time-frequency transformation; or, the multiple sub-signals are multiple sub-band signals, which are obtained by filtering the current frame audio signal.

[0012] In one possible implementation, a code rate is allocated to the plurality of sub-signals based on a first code rate. Based on the code rate allocated to the first type of sub-signals, an AI encoding module is used to encode the first type of sub-signals. Based on the code rate allocated to the second type of sub-signals, a non-AI encoding module is used to encode the second type of sub-signals.

[0013] The bitstream corresponding to the audio signal of the current frame typically includes a header and a payload. The bitrate of the header is usually fixed, while the payload carries the encoded data corresponding to the multiple sub-signals. Therefore, the bitrate of the header can be subtracted from the first bitrate to obtain the bitrate of the payload. Then, bitrates are allocated to the multiple sub-signals based on the bitrate of the payload, so that the sum of the allocated bitrates for the multiple sub-signals equals the bitrate of the payload.

[0014] Because AI encoding modules have powerful compression capabilities, while non-AI encoding modules degrade audio quality at lower bitrates, when allocating bitrates for these multiple sub-signals, a lower bitrate can be assigned to the first type of sub-signal, and a higher bitrate to the second type of sub-signal. In other words, the bitrate assigned to the first type of sub-signal is lower than that assigned to the second type of sub-signal. This ensures the audio quality of the second type of sub-signal encoded by the non-AI encoding module.

[0015] In one possible implementation, the bitstream further includes: sub-signal segmentation information and sub-signal encoding information of the current frame audio signal. The sub-signal segmentation information is used to indicate the segmentation method of the multiple sub-signals, and the sub-signal encoding information is used to indicate the encoding module corresponding to the multiple sub-signals, that is, to indicate whether each sub-signal in the multiple sub-signals is encoded by an AI encoding module or a non-AI encoding module.

[0016] In one possible implementation, the bitstream also includes: the frame state of the current frame audio signal, which indicates that the current frame audio signal is a mixed-coded frame.

[0017] In other words, if the bitstream corresponding to the current frame audio signal includes data encoded by the hybrid encoder but does not include data encoded by other encoders, then the current frame audio signal is determined to be a hybrid encoded frame.

[0018] In one possible implementation, the plurality of encoders further includes a second encoder, which is the encoder used for the previous frame audio signal. The method further includes: allocating a bitrate to the first encoder and the second encoder based on a first bitrate; encoding the current frame audio signal using the second encoder to obtain second encoded data based on the bitrate allocated to the second encoder, the bitstream further including the second encoded data; encoding a first type of sub-signal among the plurality of sub-signals using the AI ​​encoding module, and encoding a second type of sub-signal among the plurality of sub-signals using the non-AI encoding module, including: encoding the first type of sub-signal using the AI ​​encoding module and encoding the second type of sub-signal using the non-AI encoding module based on the bitrate allocated to the first encoder.

[0019] In one possible implementation, the second encoder supports a bitrate lower than that supported by the first encoder. The bitrate is allocated to the first encoder and the second encoder based on the first bitrate, including: allocating a second bitrate to the second encoder, the second bitrate being the encoding bitrate of the previous frame audio signal; and allocating a third bitrate to the first encoder, the third bitrate being the difference between the bitrate corresponding to the payload in the bitstream and the second bitrate, the bitrate corresponding to the payload being determined based on the first bitrate.

[0020] If the bitrate supported by the second encoder is lower than that supported by the first encoder, it indicates that the current frame audio signal has switched from a low bitrate to a high bitrate. In this case, a second bitrate is assigned to the second encoder, and a third bitrate is assigned to the first encoder. That is, the second encoder encodes the current frame audio signal at the same bitrate as the previous frame audio signal, while the first encoder uses the remaining bitrate to encode the current frame audio signal. This avoids fluctuations in the sound quality of the current frame audio signal relative to the previous frame audio signal.

[0021] In one possible implementation, the second encoder is an AI encoder, and the bitstream also includes: the frame state of the current frame audio signal, which indicates that the current frame audio signal is a switching frame from AI encoding to hybrid encoding.

[0022] When the second encoder is an AI encoder, the bitstream corresponding to the audio signal of the current frame also includes the bitrate assigned to the second encoder.

[0023] In one possible implementation, the second encoder supports a bitrate greater than that supported by the first encoder. Assigning bitrates to the first encoder and the second encoder includes: assigning a fourth bitrate to the second encoder, the fourth bitrate being the lowest bitrate supported by the second encoder; and assigning a fifth bitrate to the first encoder, the fifth bitrate being the difference between the bitrate corresponding to the payload in the bitstream and the fourth bitrate, the bitrate corresponding to the payload being determined based on the first bitrate.

[0024] If the bitrate supported by the second encoder is greater than that supported by the first encoder, it indicates that the current frame audio signal has switched from a high bitrate to a low bitrate. In this case, a fourth bitrate is assigned to the second encoder, and a fifth bitrate is assigned to the first encoder. That is, the second encoder uses its lowest supported bitrate to encode the current frame audio signal, while the first encoder uses the remaining bitrate. This ensures that the data encoded by the second encoder can be transmitted to the decoding end, guaranteeing the continuity of the audio data.

[0025] It should be noted that allocating bitrates to the first and second encoders in the above manner, when the bitrate supported by the second encoder is greater than that supported by the first encoder, is only an example. In practical applications, other methods can also be used to allocate bitrates to the first and second encoders. For example, bitrates can be allocated to the first and second encoders such that the sum of the allocated bitrates equals a third reference bitrate. This third reference bitrate is greater than or equal to the bitrate corresponding to the payload in the current frame's audio stream, and less than or equal to the bitrate corresponding to the payload in the previous frame's audio stream. In other words, instead of directly switching to the first bitrate, any bitrate between the bitrate corresponding to the payload in the current frame's audio stream and the bitrate corresponding to the payload in the previous frame's audio stream is used as a transition, avoiding large fluctuations caused by a direct reduction in bitrate.

[0026] In one possible implementation, the second encoder is a non-AI encoder, and the bitstream also includes: the frame state of the current frame audio signal, which indicates that the current frame audio signal is a switching frame from non-AI encoding to hybrid encoding.

[0027] In one possible implementation, the bitstream also includes the size of the encoded result of the current frame audio signal, which includes the size of either the first encoded data or the second encoded data. That is, carrying the size of either the first or second encoded data in the bitstream facilitates the decoding end in parsing the first and second encoded data from the payload portion of the bitstream.

[0028] In one possible implementation, the frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal. Conversely, the frequency corresponding to the first type of sub-signal is higher than the frequency corresponding to the second type of sub-signal. That is, an AI encoding module is used to encode the low-frequency sub-signals among the multiple sub-signals, and a non-AI encoding module is used to encode the high-frequency sub-signals among the multiple sub-signals; or, an AI encoding module is used to encode the high-frequency sub-signals among the multiple sub-signals, and a non-AI encoding module is used to encode the low-frequency sub-signals among the multiple sub-signals.

[0029] Because AI encoding modules have powerful compression capabilities, the bitrate required for encoding audio signals using AI encoding modules is usually low, supporting bitrates below 8kbps, and even compressing narrowband audio signals to below 1kbps. However, the compression complexity is usually high, and the way AI encoding modules encode high-frequency signals is usually generative, which may lead to differences between the decoded spectrum and the original spectrum. Non-AI encoding modules generally have lower compression complexity, but at low bitrates, using non-AI encoding modules to encode audio signals will reduce the audio quality. Moreover, low-frequency signals are usually more important to human hearing, while high-frequency signals are relatively sparse. Therefore, using AI encoding modules to encode the low-frequency sub-signals among multiple sub-signals and using non-AI encoding modules to encode the high-frequency sub-signals among multiple sub-signals allows us to leverage the powerful compression capabilities of AI encoding modules to improve the audio quality and the low complexity of non-AI encoding modules to control computational complexity. In this way, audio quality is improved while controlling computational complexity, overcoming the problem of poor audio quality at low bitrates in traditional audio encoding and decoding.

[0030] When the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is a hybrid encoder and the second encoder is a non-AI encoder, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-packet header, a first sub-payload, and a second sub-payload. The first sub-packet header is used to carry the sub-signal division information and sub-signal encoding information of the current frame audio signal, as well as the bit rate allocated to each sub-signal in the first type of sub-signal. The first sub-payload is used to carry the first encoded data, and the second sub-payload is used to carry the second encoded data.

[0031] Similarly, when the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is a hybrid encoder and the second encoder is an AI encoder, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-packet header, a first sub-payload, a second sub-packet header, and a second sub-payload. The first sub-packet header is used to carry the sub-signal division information and sub-signal encoding information of the current frame audio signal, as well as the bit rate allocated to each sub-signal in the first type of sub-signal. The first sub-payload is used to carry the first encoded data. The second sub-packet header is used to carry the bit rate allocated to the second encoder. The second sub-payload is used to carry the second encoded data.

[0032] It should be understood that the above is only an example. The information carried by the first sub-header and the second sub-header can also be carried entirely through the header. In this case, the payload includes the first sub-payload and the second sub-payload.

[0033] After the encoding end determines the bitstream corresponding to the current frame audio signal according to the above content, the bitstream corresponding to the current frame audio signal can be transmitted to the decoding end. One or more channels can be established between the encoding and decoding ends. If one channel is established between the encoding and decoding ends, the bitstream corresponding to the current frame audio signal can be directly transmitted to the decoding end through this channel. If multiple channels are established between the encoding and decoding ends, the bitstream corresponding to the current frame audio signal can be divided into multiple sub-bitstreams, and then transmitted to the decoding end through these multiple channels.

[0034] In one possible implementation, high-importance sub-streams can be transmitted through high-priority channels, while low-importance sub-streams can be transmitted through low-priority channels. If all these channels are in good condition and the sub-streams transmitted through them are received by the decoder, the decoder can combine these sub-streams to obtain the bitstream corresponding to the current frame's audio signal, and then perform decoding as described below. However, if the low-priority channels are in poor condition, the sub-streams transmitted through high-priority channels can at least guarantee audio intelligibility.

[0035] In one possible implementation, the plurality of sub-bitstreams includes a first sub-bitstream and a second sub-bitstream. The first sub-bitstream includes the bitstream of the low-frequency portion of the current frame audio signal, and the second sub-bitstream includes the bitstream of the high-frequency portion of the current frame audio signal. Alternatively, the second sub-bitstream includes the bitstream of the entire current frame audio signal. The first sub-bitstream is of higher importance than the second sub-bitstream.

[0036] If the second sub-stream includes the entire audio signal of the current frame, it means that the first sub-stream is transmitted repeatedly. That is, the highly important sub-stream is transmitted redundantly. This not only ensures the stability of the connection, but also ensures the continuity of listening, and greatly improves the anti-interference capability of the channel.

[0037] Secondly, an audio decoding method is provided, comprising: determining a first decoder from a plurality of decoders for decoding the audio signal of the current frame; if the first decoder is a hybrid decoder, then using the hybrid decoder to parse the bitstream corresponding to the audio signal of the current frame to obtain multiple reconstructed sub-signals; wherein the hybrid decoder includes an AI decoding module and a non-AI decoding module, the multiple reconstructed sub-signals include reconstructed first-type sub-signals and reconstructed second-type sub-signals, the reconstructed first-type sub-signals are obtained by parsing through the AI ​​decoding module, and the reconstructed second-type sub-signals are obtained by parsing through the non-AI decoding module; and obtaining the reconstructed audio signal of the current frame based on the multiple reconstructed sub-signals.

[0038] AI encoding modules offer powerful compression capabilities, but their compression complexity is typically high. Non-AI encoding modules, while generally less complex, suffer from reduced audio quality at low bitrates. Therefore, a combination of AI and non-AI encoding modules is employed. AI encoding modules encode a subset of the multiple sub-signals, while non-AI modules encode the remaining subset. This results in higher audio quality for the first type of sub-signals decoded by the AI ​​decoding module, and lower complexity for the second type. This approach improves audio quality while controlling computational complexity, addressing the issue of poor audio quality at low bitrates in traditional audio encoding and decoding.

[0039] In one possible implementation, the hybrid decoder is used to parse the bitstream corresponding to the audio signal of the current frame to obtain multiple reconstructed sub-signals, including: obtaining sub-signal segmentation information and sub-signal encoding information based on the bitstream, wherein the sub-signal segmentation information is used to indicate the segmentation method of the multiple sub-signals, and the sub-signal encoding information is used to indicate the encoding module corresponding to the multiple sub-signals; and the hybrid decoder is used to parse the bitstream based on the sub-signal segmentation information and the sub-signal encoding information to obtain the multiple reconstructed sub-signals.

[0040] Since sub-signal segmentation information indicates the segmentation method of multiple sub-signals, and sub-signal encoding information indicates the encoding module corresponding to multiple sub-signals (i.e., whether each sub-signal is encoded using an AI encoding module or a non-AI encoding module), after obtaining the sub-signal segmentation information, the encoded data of each sub-signal can be distinguished from the bitstream corresponding to the current frame audio signal. Then, according to the encoding module corresponding to the multiple sub-signals, the bitstream corresponding to the current frame audio signal is parsed to obtain the reconstructed multiple sub-signals.

[0041] As an example, the encoding module corresponding to the first sub-signal is an AI encoding module. An AI decoding module is used to parse the encoded data of the first sub-signal in the bitstream corresponding to the current frame audio signal to obtain the reconstructed first sub-signal. The first sub-signal is any one of the multiple sub-signals corresponding to the AI ​​encoding module. The encoding module corresponding to the second sub-signal is a non-AI encoding module. A non-AI decoding module is used to parse the encoded data of the second sub-signal to obtain the reconstructed second sub-signal. The second sub-signal is any one of the multiple sub-signals corresponding to the non-AI encoding module.

[0042] In one possible implementation, determining a first decoder from a plurality of decoders includes: obtaining the frame state of the current frame audio signal based on the bitstream; and determining the first decoder from the plurality of decoders based on the frame state.

[0043] Frame state indication includes three independent encoded frame states: AI-encoded frame, hybrid-encoded frame, and non-AI-encoded frame; and six switching frame types: AI to non-AI, non-AI to AI, non-AI to hybrid, hybrid to non-AI, AI to hybrid, and hybrid to AI. Multiple decoders include AI decoders, hybrid decoders, and non-AI decoders. If the frame state of the current frame audio signal indicates that the current frame audio signal is an AI-encoded frame, then the AI ​​decoder among the multiple decoders is determined as the first decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a hybrid-encoded frame, then the hybrid decoder among the multiple decoders is determined as the first decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a non-AI-encoded frame, then the non-AI decoder among the multiple decoders is determined as the first decoder.

[0044] If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from AI to non-AI, then the non-AI decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the AI ​​decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from non-AI to AI, then the AI ​​decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the non-AI decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from non-AI to mixed, then the mixed decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the non-AI decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from mixed to non-AI, then the non-AI decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the mixed decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from AI to mixed, then the mixed decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the AI ​​decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a switching frame from mixing to AI, then the AI ​​decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the mixing decoder.

[0045] In one possible implementation, the plurality of decoders further includes a second decoder. The bitstream includes first encoded data and second encoded data. The first encoded data is obtained by encoding the current frame audio signal by a first encoder, and the second encoded data is obtained by encoding the current frame audio signal by a second encoder. The reconstructed multiple sub-signals are obtained by parsing the first encoded data. Based on the reconstructed multiple sub-signals, a reconstructed current frame audio signal is obtained, including: determining a first reconstructed audio signal based on the reconstructed multiple sub-signals; parsing the bitstream using the second decoder to obtain a second reconstructed audio signal, wherein the second decoder is the decoder used for the previous frame audio signal; and obtaining the reconstructed current frame audio signal based on the first reconstructed audio signal and the second reconstructed audio signal.

[0046] In one possible implementation, a second decoder is used to parse the bitstream to obtain a second reconstructed audio signal, including: obtaining the size of the encoding result of the current frame audio signal based on the bitstream, the size of the encoding result including the size of the first encoded data or the second encoded data; and using the second decoder to parse the bitstream based on the size of the encoding result to obtain the second reconstructed audio signal.

[0047] In one possible implementation, the frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal.

[0048] In one possible implementation, an audio smoothing strategy is employed to smooth the first and second reconstructed audio signals to obtain the reconstructed current frame audio signal.

[0049] As an example, the end of the second reconstructed audio signal and the beginning of the first reconstructed audio signal are smoothed. The beginning of the second reconstructed audio signal is used as the beginning of the reconstructed current frame audio signal, and the smoothed portion is used as the end of the reconstructed current frame audio signal, resulting in the reconstructed current frame audio signal. In this way, the reconstructed current frame audio signal gradually transitions from the previous frame to the current frame, ensuring seamless switching between the audio data parsed by the two decoders.

[0050] Thirdly, an encoding device is provided, which has the functionality to implement the method behavior described in the first aspect. The encoding device includes at least one module for implementing the method provided in the first aspect.

[0051] Fourthly, a decoding device is provided, which has the functionality to implement the method behavior described in the second aspect above. The decoding device includes at least one module for implementing the method provided in the second aspect above.

[0052] Fifthly, an encoding device is provided, comprising: a processor coupled to a memory for storing a program or instructions, wherein when the program or instructions are executed by the processor, the encoding device performs the method described in the first aspect.

[0053] In a sixth aspect, a decoding device is provided, comprising: a processor coupled to a memory for storing a program or instructions which, when executed by the processor, cause the decoding device to perform the method described in the second aspect.

[0054] In a seventh aspect, a coding and decoding system is provided, which includes the encoding device of the fifth aspect and / or the decoding device of the sixth aspect.

[0055] Eighthly, a computer-readable storage medium is provided, including program code that, when executed on a computer, causes the computer to perform the method of the first aspect or the method of the second aspect described above.

[0056] Ninthly, a computer program product is provided, including instructions that, when executed on a computer, cause the computer to perform the method of the first aspect or the method of the second aspect described above.

[0057] In a tenth aspect, a computer-readable storage medium is provided, on which a bitstream obtained according to the method of the first aspect is stored.

[0058] Eleventhly, a device for storing a bitstream is provided, comprising at least one storage medium and a communication interface; the communication interface is used to receive or transmit the bitstream; the at least one storage medium is used to store the bitstream; the bitstream is encoded by an encoder according to the method of the first aspect described above.

[0059] In a twelfth aspect, a method for storing a bitstream is provided, comprising: receiving the bitstream via a communication interface; and storing the bitstream in one or more storage media, wherein the bitstream is encoded by an encoder according to the method of the first aspect described above.

[0060] In a thirteenth aspect, a system for distributing a bitstream is provided, comprising at least one storage medium and a video streaming device; the at least one storage medium is used to store a bitstream encoded by an encoder according to the method of the first aspect described above; the video streaming device is used to transmit the bitstream in the at least one storage medium to the decoder in response to a request from the decoder.

[0061] In a fourteenth aspect, a method for distributing a bitstream is provided, comprising: receiving a first request; in response to the first request, selecting a bitstream from at least one storage medium; and sending the bitstream to a destination device; wherein the at least one storage medium is used to store the bitstream, the bitstream being encoded by an encoder according to the method of the first aspect described above.

[0062] In a fifteenth aspect, a system for processing bitstreams is provided, comprising an image source device, an encoder, one or more storage media, and a destination device; the image source device is used to provide image data; the encoder is used to acquire the image data from the image source device via an interface and encode the image data to obtain one or more bitstreams, the bitstreams being encoded by the encoder according to the method of the first aspect described above; the encoder is used to store the one or more bitstreams in one or more storage media; or the encoder is used to encapsulate the one or more bitstreams to obtain a transmission bitstream; the encoder is used to transmit the transmission bitstream to the destination device via a communication link or communication network; the destination device is used to decapsulate the transmission bitstream to obtain the one or more bitstreams; and the destination device is used to decode the one or more bitstreams to obtain decoded data.

[0063] The technical effects achieved by the third to fifteenth aspects mentioned above are similar to those achieved by the corresponding technical means in the first or second aspects, and will not be repeated here. Attached Figure Description

[0064] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0065] Figure 2 is a schematic diagram of an exemplary implementation environment provided in an embodiment of this application;

[0066] Figure 3 is a flowchart of an audio encoding method provided in an embodiment of this application;

[0067] Figure 4 is a schematic diagram of a method for determining a sub-signal provided in an embodiment of this application;

[0068] Figure 5 is a schematic diagram of another method for determining sub-signals provided in an embodiment of this application;

[0069] Figure 6 is a schematic diagram of a bitstream format provided in an embodiment of this application;

[0070] Figure 7 is a schematic diagram of another bitstream format provided in an embodiment of this application;

[0071] Figure 8 is a schematic diagram of another bitstream format provided in an embodiment of this application;

[0072] Figure 9 is a schematic diagram of an encoding process provided in an embodiment of this application;

[0073] Figure 10 is a flowchart of an audio decoding method provided in an embodiment of this application;

[0074] Figure 11 is a schematic diagram of an AI encoding and decoding process provided in an embodiment of this application;

[0075] Figure 12 is a schematic diagram of an AI encoding and decoding process provided in an embodiment of this application;

[0076] Figure 13 is a schematic diagram of a non-AI encoding / decoding process provided in an embodiment of this application;

[0077] Figure 14 is a schematic diagram of an audio smoothing process provided in an embodiment of this application;

[0078] Figure 15 is a schematic diagram of switching an AI encoder to a hybrid encoder according to an embodiment of this application;

[0079] Figure 16 is a schematic diagram of a decoding process provided in an embodiment of this application;

[0080] Figure 17 is a schematic diagram of the structure of an audio encoding device provided in an embodiment of this application;

[0081] Figure 18 is a schematic diagram of the structure of an audio decoding device provided in an embodiment of this application. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0083] With the development of short-range wireless communication technologies such as Bluetooth, Wi-Fi, and Wi-Fi, as well as long-range wireless communication technologies such as Voice over Internet Protocol (VoIP), audio playback scenarios based on short-range or long-range wireless communication technologies are ubiquitous in people's lives. For example, two users making a voice call via VoIP, with the audio played through the terminal, can be considered a long-range VoIP audio playback scenario. Alternatively, a terminal connected to headphones via Bluetooth, receiving the audio from the other end and transmitting it to the headphones for playback, can be considered a short-range Bluetooth audio playback scenario. Or, a user needs to play music, and the terminal is connected to headphones via Bluetooth, with the terminal transmitting the music to the headphones for playback, which can also be considered a short-range Bluetooth audio playback scenario. The following section will use a voice call scenario as an example.

[0084] With the gradual popularization of high-definition calling, the quality of voice calls has been greatly improved. Audio codecs play a crucial role in this improvement. For example, the Opus audio codec used in long-distance calls and the LC3 audio codec used in short-range Bluetooth calls are widely integrated into consumer products. These audio codecs, based on traditional digital signal processing, whether using frequency domain, time domain, or a hybrid time-frequency domain scheme, can meet the needs of wireless voice calls in most scenarios. However, they also have some limitations in applicable scenarios. For example, in weak network communication, no network communication, and communication with high wireless interference, the transmission bit rate drops sharply, posing a significant challenge to the sound quality of voice calls. The following sections will briefly explain a few of these situations.

[0085] 1. The audio encoding bitrate of short-range Bluetooth is 64kbps. However, in high-interference scenarios such as airports and subways, the transmission bitrate decreases. Accordingly, the encoding bitrate needs to be reduced to ensure audio continuity. However, as the encoding bitrate decreases, the audio encoding quality also decreases. This may introduce audio encoding roughness and noise due to insufficient encoding bitrate, which will greatly affect people's voice call audio quality experience.

[0086] 2. To ensure sound quality, the minimum bit rate for long-distance VoIP voice calls is generally controlled at around 15kbps. However, with the instability of network links, the packet loss rate of voice calls in weak network scenarios such as high-speed rail, elevators, and parking garages increases sharply, making it difficult to guarantee people's voice call sound quality experience.

[0087] 3. In the walkie-talkie mode of outdoor network-free communication scenarios based on short-range Bluetooth protocol, low bit rate audio encoding and ultra-low bit rate audio encoding play a key role. Currently used codecs are generally ultra-low bit rate narrowband encoding, which has the characteristics of high latency and only guarantees voice intelligibility. It cannot improve the sound quality of the call, and it is also not user-friendly for two-way communication.

[0088] Based on this, this application proposes an audio encoding and decoding method. In this method, a first encoder is determined from multiple encoders based on a first bitrate, where the first bitrate is the encoding bitrate of the current frame audio signal. The first encoder is used to encode the current frame audio signal. If the first encoder is a hybrid encoder, multiple sub-signals are obtained based on the current frame audio signal. The hybrid encoder includes an artificial intelligence (AI) encoding module and a non-AI encoding module. The AI ​​encoding module encodes a first type of sub-signal among the multiple sub-signals, and the non-AI encoding module encodes a second type of sub-signal among the multiple sub-signals. That is, by combining the AI ​​encoding module and the non-AI encoding module, a portion of the multiple sub-signals are encoded by the AI ​​encoding module, and another portion of the multiple sub-signals are encoded by the non-AI encoding module. This utilizes the powerful compression capability of the AI ​​encoding module to improve the audio encoding sound quality, and utilizes the low complexity of the non-AI encoding module to control computational complexity. In this way, the audio encoding sound quality is improved while controlling computational complexity, overcoming the problem of poor sound quality in traditional audio encoding and decoding at low bitrates.

[0089] Please refer to Figure 1, which is a schematic diagram of an implementation environment provided in an embodiment of this application. This implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 can generate encoded audio. Therefore, the source device 10 can also be referred to as an audio encoding device or encoding end. The destination device 20 can decode the encoded audio generated by the source device 10. Therefore, the destination device 20 can also be referred to as an audio decoding device or decoding end. The link 30 can receive the encoded audio generated by the source device 10 and can transmit the encoded audio to the destination device 20. The storage device 40 can receive the encoded audio generated by the source device 10 and can store the encoded audio. Under these conditions, the destination device 20 can directly obtain the encoded audio from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded audio generated by the source device 10. Under these conditions, the destination device 20 can stream or download the encoded audio stored in the storage device 40.

[0090] Both source device 10 and destination device 20 include one or more processors and memory coupled to the one or more processors. This memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures. Both source device 10 and destination device 20 may include mobile phones, smartphones, personal digital assistants (PDAs), wearable devices, pocket PCs (PPCs), tablets, smart car systems, smart TVs, smart speakers, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0091] Link 30 may include one or more media or devices capable of transmitting encoded audio from source device 10 to destination device 20. In one possible implementation, link 30 may include one or more communication media enabling source device 10 to directly transmit encoded audio to destination device 20 in real time. In embodiments of this application, source device 10 may modulate the encoded audio based on a communication standard, such as a wireless communication protocol, and transmit the modulated audio to destination device 20. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices facilitating communication from source device 10 to destination device 20.

[0092] In one possible implementation, storage device 40 can store the received encoded audio transmitted by source device 10, and destination device 20 can directly retrieve the encoded audio from storage device 40. Under such conditions, storage device 40 can include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded audio.

[0093] In one possible implementation, storage device 40 may correspond to a file server or an intermediate storage device capable of storing encoded audio generated by source device 10, and destination device 20 may stream or download the audio stored on storage device 40. The file server may be any type of server capable of storing encoded audio and sending it to destination device 20. In one possible implementation, the file server may include a web server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. Destination device 20 may acquire the encoded audio via any standard data connection (including an Internet connection). Any standard data connection may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for acquiring encoded audio stored on a file server. The transmission of encoded audio from storage device 40 may be streaming, downloading, or a combination of both.

[0094] The implementation environment shown in Figure 1 is only one possible implementation method, and the technology of this application embodiment can be applied not only to the source device 10 that can encode audio and the destination device 20 that can decode the encoded audio shown in Figure 1, but also to other devices that can encode audio and decode the encoded audio.

[0095] In the implementation environment shown in Figure 1, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, output interface 140 may include a modem / demodulator and / or a transmitter, wherein the transmitter may also be referred to as a transmitter. Data source 120 may include an audio capture device (e.g., a camera, etc.), an archive containing previously captured audio, a feed interface for receiving audio from an audio content provider, and / or a computer system for generating audio, or a combination of these sources of audio.

[0096] Data source 120 can send audio to encoder 100, which can encode the received audio from data source 120 to obtain encoded audio. The encoder can then send the encoded audio to an output interface. In some embodiments, source device 10 directly sends the encoded audio to destination device 20 via output interface 140. In other embodiments, the encoded audio can also be stored on storage device 40 for later retrieval by destination device 20 for decoding and / or playback.

[0097] In the implementation environment shown in Figure 1, the destination device 20 includes an input interface 240, a decoder 200, and a playback device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive encoded audio via link 30 and / or from storage device 40, and then send it to the decoder 200, which may decode the received encoded audio to obtain decoded audio. The decoder may send the decoded audio to the playback device 220. The playback device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the playback device 220 plays the decoded audio.

[0098] Encoder 100 and decoder 200 may each be any of the following: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques of the embodiments of this application are implemented in part in software, the apparatus may store instructions for software in a suitable non-volatile computer-readable storage medium, and the instructions may be executed in hardware using one or more processors to implement the techniques of the embodiments of this application. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors.

[0099] Figure 2 is a schematic diagram of an exemplary implementation environment provided by an embodiment of this application. Referring to Figure 2, the implementation environment includes an audio transmitting device and an audio receiving device. The audio transmitting device can be the source device in Figure 1 above, and the audio transmitting device is configured with an audio encoder. The audio receiving device can be the destination device in Figure 1 above, and the audio receiving device is configured with an audio decoder. The audio transmitting device can be a mobile phone, computer, tablet, etc. Among them, the computer can be a laptop computer, desktop computer, etc., and the tablet can be a handheld tablet, in-vehicle tablet, etc. The audio receiving device can be a true wireless stereo (TWS) headset, smart screen, smart speaker, wireless headphones, wireless neckband headphones, smartwatch, smart glasses, smart in-vehicle device, etc. In some other embodiments, the audio receiving device can also be a mobile phone, computer, tablet, etc.

[0100] It should be noted that the application scenarios and implementation environments described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, as the application scenarios and implementation environments evolve, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0101] Figure 3 is a flowchart of an audio encoding method provided in an embodiment of this application. The method is applied to an encoding end, which can be the source device in Figure 1 or the audio transmission device in Figure 2. Referring to Figure 3, the method includes steps 301-303.

[0102] Step 301: Determine the first encoder from multiple encoders based on the first bit rate, where the first bit rate is the encoding bit rate of the audio signal of the current frame, and the first encoder is used to encode the audio signal of the current frame.

[0103] The first bit rate is determined based on a number of influencing factors. For example, these factors include the current environment of the encoder, i.e., whether the encoder is in a low-interference or high-interference environment; the received signal strength indication (RSSI) of the encoder and decoder; and if the encoder and decoder are connected by short-range wireless communication technology, the distance between the encoder and decoder is also a factor.

[0104] In some embodiments, each of the plurality of encoders corresponds to a bitrate range, which is the bitrate range supported by the corresponding encoder. The encoder corresponding to the bitrate range containing the first bitrate among the plurality of encoders is defined as the first encoder. That is, multiple encoders are deployed at the encoding end, with different encoders corresponding to different bitrate ranges. In this way, as the encoding bitrate dynamically changes, the corresponding encoder can be selected to encode the audio signal, thereby adapting to different scenarios and enabling continuous transmission of the audio signal.

[0105] As an example, the multiple encoders include an AI encoder, a hybrid encoder, and a non-AI encoder. The hybrid encoder includes both AI and non-AI encoding modules. The bitrate within the bitrate range corresponding to the AI ​​encoder is lower than the bitrate within the bitrate range corresponding to the hybrid encoder, and the bitrate within the bitrate range corresponding to the hybrid encoder is lower than the bitrate within the bitrate range corresponding to the non-AI encoder.

[0106] It should be understood that an AI encoder refers to an encoder that uses AI encoding technology, a hybrid encoder refers to an encoder that uses both AI and non-AI encoding technologies, and a non-AI encoder refers to an encoder that uses only non-AI encoding technology. Non-AI encoding technology can be understood as traditional encoding technology. Furthermore, the AI ​​encoding technology used by an AI encoder may be the same as or different from that used by a hybrid encoder, and the non-AI encoding technology used by a non-AI encoder may be the same as or different from that used by a hybrid encoder; this application does not limit the scope of this application.

[0107] In some embodiments, the bitrate range corresponding to the AI ​​encoder is less than or equal to the first reference bitrate. The bitrate range corresponding to the hybrid encoder is greater than the first reference bitrate and less than the second reference bitrate. The bitrate range corresponding to the non-AI encoder is greater than or equal to the second reference bitrate. The first reference bitrate is less than the second reference bitrate. For example, the first reference bitrate is 8kbps, 10kbps, 12kbps, 14kbps, or 16kbps, and the second reference bitrate is 32kbps.

[0108] Non-AI encoders can be any audio encoder suitable for wireless audio compression and transmission scenarios, including but not limited to: continuously variable slope delta modulation (CVSD) encoders, sub-band coding (SBC) encoders, and Bluetooth Advanced Audio Encoder from the Moving Picture Experts Group (MPEG). Audio coding, AAC series (such as AAC-LC, AAC-LD, AAC-HE, AAC-HEv2, etc.), aptX series (such as aptX, aptX HD, aptX low latency) encoders, low-latency and high-definition audio codecs. High-definition audio codec (LHDC), low-power, low-latency LC3 audio codec, and LC3plus, etc.

[0109] The current frame audio signal can be in Pulse Code Modulation (PCM) format. The data content can be speech or a speech signal mixed with music. The current frame audio signal can be mono, stereo, or multi-channel data. The bit depth can be 16-bit, 24-bit, 32-bit floating-point, or 32-bit fixed-point. Supported sampling rates include 44.1 kHz, 48 kHz, 88.2 kHz, and 96 kHz. Different sampling rates are used in different application scenarios; for example, 8 kHz, 16 kHz, and 32 kHz are commonly used for short-range voice calls, while 16 kHz and 48 kHz are commonly used for long-range VoIP voice calls.

[0110] Step 302: If the first encoder is a hybrid encoder, then multiple sub-signals are obtained based on the current frame audio signal. The hybrid encoder includes an AI encoding module and a non-AI encoding module.

[0111] In some embodiments, the multiple sub-signals are multiple frequency band signals, which are obtained by dividing the spectrum obtained after time-frequency transformation of the current frame audio signal. That is, as shown in Figure 4, time-frequency transformation is performed on the current frame audio signal to obtain the spectrum, and the spectrum is divided to obtain multiple frequency band signals, which are the multiple sub-signals.

[0112] There are several ways to perform time-frequency transformation on the current frame's audio signal, such as the Fast Fourier Transform (FFT). Fourier transformation (FFT), modified discrete cosine transform (MDCT), integer modified discrete cosine transform (INT-MDCT), etc.

[0113] There are various ways to divide the spectrum. For example, according to frequency, the spectrum can be divided into two frequency bands: low frequency and high frequency. Alternatively, it can be divided into three, four, six, or eight frequency bands. This application does not limit the method of dividing the frequency bands.

[0114] In other embodiments, the plurality of sub-signals are plurality of sub-band signals, which are obtained by filtering the current frame audio signal. That is, as shown in Figure 5, filtering the current frame audio signal yields plurality of sub-band signals, which are the plurality of sub-signals.

[0115] There are several ways to filter the audio signal of the current frame, such as quadrature mirror filters (QMF) and pseudo-quadrature mirror filters (PQMF).

[0116] These multiple sub-band signals are obtained by filtering the current frame audio signal. In other words, the division of these multiple sub-band signals depends on the filtering method of the current frame audio signal. Different filtering methods will result in different multiple sub-band signals. For example, filtering the current frame audio signal may yield two sub-band signals, one for low frequency and one for high frequency, or three, four, six, eight, etc., sub-band signals after frequency filtering.

[0117] Step 303: Use an AI encoding module to encode the first type of sub-signal among the multiple sub-signals, and use a non-AI encoding module to encode the second type of sub-signal among the multiple sub-signals to obtain the first encoded data. The bitstream corresponding to the audio signal of the current frame includes the first encoded data.

[0118] The frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal, or the frequency corresponding to the first type of sub-signal is higher than the frequency corresponding to the second type of sub-signal. That is to say, the low-frequency sub-signal among the multiple sub-signals is encoded using an AI encoding module and the high-frequency sub-signal among the multiple sub-signals is encoded using a non-AI encoding module, or the high-frequency sub-signal among the multiple sub-signals is encoded using an AI encoding module and the low-frequency sub-signal among the multiple sub-signals is encoded using a non-AI encoding module. This application embodiment does not limit this.

[0119] AI encoding modules possess powerful compression capabilities (for the same data, the encoding bitrate required by AI encoding modules is approximately one-third that of non-AI encoding modules; that is, the AI ​​encoding effect at a bitrate of 6kbps is comparable to the non-AI encoding effect at a bitrate of around 15kbps). Using AI encoding modules to encode audio signals typically requires a lower bitrate, supporting bitrates below 8kbps, and even compressing narrowband audio signals to below 1kbps. However, the compression complexity is usually higher, and the way AI encoding modules encode high-frequency signals is often generative, leading to potential differences between the decoded and original spectra. Non-AI encoding modules generally have lower compression complexity, but at lower bitrates, using them to encode audio signals degrades the audio quality. Furthermore, low-frequency signals are generally more important to human hearing, while high-frequency signals are relatively sparse. Therefore, using AI encoding modules to encode the low-frequency sub-signals and non-AI encoding modules to encode the high-frequency sub-signals leverages the powerful compression capabilities of AI to improve audio quality and utilizes the lower complexity of non-AI encoding modules to control computational complexity. In this way, the audio encoding quality can be improved while controlling the computational complexity, thus making up for the problem of poor audio quality at low bit rates in traditional audio encoding and decoding.

[0120] It should be noted that the algorithm latency of the AI ​​encoding module and the non-AI encoding module can be the same, thus enabling the encoding of sub-signals in parallel using both modules. Of course, the algorithm latency of the AI ​​encoding module and the non-AI encoding module can also be different; this embodiment does not limit this.

[0121] In some embodiments, a code rate is allocated to the plurality of sub-signals based on a first code rate, an AI encoding module is used to encode the first type of sub-signals based on the code rate allocated to the first type of sub-signals, and a non-AI encoding module is used to encode the second type of sub-signals based on the code rate allocated to the second type of sub-signals.

[0122] The bitstream corresponding to the audio signal of the current frame typically includes a header and a payload. The bitrate of the header is usually fixed, while the payload carries the encoded data corresponding to the multiple sub-signals. Therefore, the bitrate of the header can be subtracted from the first bitrate to obtain the bitrate of the payload. Then, bitrates are allocated to the multiple sub-signals based on the bitrate of the payload, so that the sum of the allocated bitrates for the multiple sub-signals equals the bitrate of the payload.

[0123] In some cases, the bitstream corresponding to the audio signal of the current frame may also include side information, which may be located in the packet header or outside the packet header. When the side information is outside the packet header, the bitrate corresponding to the packet header and the bitrate corresponding to the side information are subtracted from the first bitrate to obtain the bitrate corresponding to the payload. The bitrate corresponding to the payload is then used to allocate bitrates for these multiple sub-signals. The following explanation will use an example where the bitstream includes both a packet header and a payload.

[0124] Because AI encoding modules have powerful compression capabilities, while non-AI encoding modules degrade audio quality at lower bitrates, when allocating bitrates for these multiple sub-signals, a lower bitrate can be assigned to the first type of sub-signal, and a higher bitrate to the second type of sub-signal. In other words, the bitrate assigned to the first type of sub-signal is lower than that assigned to the second type of sub-signal. This ensures the audio quality of the second type of sub-signal encoded by the non-AI encoding module.

[0125] AI encoding modules typically include one or more encoding networks, each supporting one or more bitrates. The process of encoding a first type of sub-signal using an AI encoding module, based on the bitrate assigned to it, includes: determining a first encoding network from the one or more encoding networks, where the bitrate supported by the first encoding network is the same as the bitrate assigned to the first sub-signal; the first sub-signal being any one of the first type of sub-signals; inputting the first sub-signal into the first encoding network to obtain an audio feature vector output by the first encoding network; quantizing the audio feature vector to obtain the quantization result; and encoding the quantization result to obtain the encoded data of the first sub-signal.

[0126] There are several ways to quantize the audio feature vector, such as vector quantization (VQ), scalar quantization (SQ), and residual vector quantization (RVQ).

[0127] In some embodiments, when the AI ​​encoding module is used to encode the first type of sub-signals, the bitstream corresponding to the audio signal of the current frame also includes the bitrate allocated to each sub-signal in the first type of sub-signals.

[0128] It should be noted that the audio feature vector includes one or more feature vectors, and each feature vector is encoded in the same way. This application embodiment takes an audio feature vector including one feature vector as an example for introduction.

[0129] The AI ​​encoding module includes an encoding network that can be a deep neural network, which includes multiple residual convolutional layers. Of course, it can also be other types of networks, but this application embodiment does not limit this.

[0130] The process of encoding the second type of sub-signal using a non-AI encoding module, based on the assigned code rate, includes: applying an analysis window and performing time-frequency transformation on the second sub-signal within the second type of sub-signal to obtain its spectrum; the second sub-signal being any one of the sub-signals in the second type of sub-signal; quantizing the spectrum of the second sub-signal based on the assigned code rate; and encoding the quantization result to obtain the encoded data of the second sub-signal.

[0131] In some embodiments, the bitstream corresponding to the current frame audio signal further includes sub-signal partitioning information and sub-signal encoding information of the current frame audio signal. The sub-signal partitioning information is used to indicate the partitioning method of the multiple sub-signals, and the sub-signal encoding information is used to indicate the encoding module corresponding to the multiple sub-signals, that is, to indicate whether each sub-signal in the multiple sub-signals is encoded by an AI encoding module or a non-AI encoding module.

[0132] In some embodiments, the bitstream corresponding to the current frame audio signal also includes the frame state of the current frame audio signal, which indicates that the current frame audio signal is a hybrid coded frame. That is, if the bitstream corresponding to the current frame audio signal includes data encoded by a hybrid encoder but does not include data encoded by other encoders, then the current frame audio signal is determined to be a hybrid coded frame.

[0133] It should be noted that, as shown in Figure 6, the bitstream corresponding to the current frame audio signal includes a header and a payload. The header carries the sub-signal division information and sub-signal encoding information of the current frame audio signal, the frame state of the current frame audio signal, and the bitrate allocated to each sub-signal in the first type of sub-signals. The header also carries information such as the sampling rate, number of channels, and frame length. The payload carries the first encoded data.

[0134] Steps 302 and 303 above describe the case where a hybrid encoder is used alone to encode the current frame audio signal. In other embodiments, the plurality of encoders also includes a second encoder, which is the encoder used for the previous frame audio signal, and the first encoder is different from the second encoder. Thus, the current frame audio signal is encoded not only by the first encoder but also by the second encoder, where the first encoder is a hybrid encoder and the second encoder is not. The following steps will describe this further.

[0135] (1) The first code rate is used to allocate the code rate to the first encoder and the second encoder.

[0136] In some embodiments, the bitrate supported by the second encoder is less than the bitrate supported by the first encoder. In this case, the implementation process of allocating bitrates based on the first bitrate to the first encoder and the second encoder includes: allocating a second bitrate to the second encoder, the second bitrate being the encoding bitrate of the previous frame audio signal; allocating a third bitrate to the first encoder, the third bitrate being the difference between the bitrate corresponding to the payload in the bitstream corresponding to the current frame audio signal and the second bitrate, the bitrate corresponding to the payload being determined based on the first bitrate.

[0137] Based on the above description, a bitstream typically includes a header and a payload. The bitrate corresponding to the header is usually fixed, while the payload carries the encoded data obtained by the first encoder and the second encoder for the current frame's audio signal. Therefore, the bitrate corresponding to the payload can be obtained by subtracting the bitrate corresponding to the header from the first bitrate. Then, the bitrate is allocated to the first and second encoders based on the bitrate corresponding to the payload.

[0138] If the bitrate supported by the second encoder is lower than that supported by the first encoder, it indicates that the current frame audio signal has switched from a low bitrate to a high bitrate. In this case, a second bitrate is assigned to the second encoder, and a third bitrate is assigned to the first encoder. That is, the second encoder encodes the current frame audio signal at the same bitrate as the previous frame audio signal, while the first encoder uses the remaining bitrate to encode the current frame audio signal. This avoids fluctuations in the sound quality of the current frame audio signal relative to the previous frame audio signal.

[0139] Based on the description in step 301, when the multiple encoders include AI encoders, hybrid encoders, and non-AI encoders, if the bitrate supported by the second encoder is less than the bitrate supported by the first encoder, and the first encoder is a hybrid encoder, then the second encoder is determined to be an AI encoder. That is, the current switch is from an AI encoder to a hybrid encoder.

[0140] In other embodiments, the second encoder supports a bitrate higher than that supported by the first encoder. In this case, the process of allocating bitrates to the first and second encoders includes: allocating a fourth bitrate to the second encoder, where the fourth bitrate is the lowest bitrate supported by the second encoder; and allocating a fifth bitrate to the first encoder, where the fifth bitrate is the difference between the bitrate corresponding to the payload in the bitstream corresponding to the current frame audio signal and the fourth bitrate. Alternatively, the first encoder may be allocated a sixth bitrate, where the sixth bitrate is the lowest bitrate supported by the first encoder; and the second encoder may be allocated a seventh bitrate, where the seventh bitrate is the difference between the bitrate corresponding to the payload in the bitstream corresponding to the current frame audio signal and the sixth bitrate.

[0141] If the bitrate supported by the second encoder is greater than that supported by the first encoder, it indicates that the current frame audio signal has switched from a high bitrate to a low bitrate. In this case, a fourth bitrate is assigned to the second encoder, and a fifth bitrate is assigned to the first encoder. That is, the second encoder uses its lowest supported bitrate to encode the current frame audio signal, while the first encoder uses the remaining bitrate. This ensures that the data encoded by the second encoder can be transmitted to the decoding end, guaranteeing the continuity of the audio data.

[0142] Similarly, a sixth bitrate is assigned to the first encoder, and a seventh bitrate is assigned to the second encoder. This means the first encoder uses its lowest supported bitrate to encode the current frame's audio signal, while the second encoder uses the remaining bitrate. This ensures that the data encoded by the first encoder can be transmitted to the decoding end, guaranteeing the continuity of the audio data.

[0143] Based on the description in step 301, when the multiple encoders include AI encoders, hybrid encoders, and non-AI encoders, if the bitrate supported by the second encoder is greater than the bitrate supported by the first encoder, and the first encoder is a hybrid encoder, then the second encoder is determined to be a non-AI encoder. That is, the current operation switches from a non-AI encoder to a hybrid encoder.

[0144] It should be noted that allocating bitrates to the first and second encoders in the above manner, when the bitrate supported by the second encoder is greater than that supported by the first encoder, is only an example. In practical applications, other methods can also be used to allocate bitrates to the first and second encoders. For example, bitrates can be allocated to the first and second encoders such that the sum of the allocated bitrates equals a third reference bitrate. This third reference bitrate is greater than or equal to the bitrate corresponding to the payload in the current frame's audio stream, and less than or equal to the bitrate corresponding to the payload in the previous frame's audio stream. In other words, instead of directly switching to the first bitrate, any bitrate between the bitrate corresponding to the payload in the current frame's audio stream and the bitrate corresponding to the payload in the previous frame's audio stream is used as a transition, avoiding large fluctuations caused by a direct reduction in bitrate.

[0145] (2) Based on the bit rate allocated to the second encoder, the current frame audio signal is encoded by the second encoder to obtain the second encoded data. The bit stream of the current frame audio signal also includes the second encoded data.

[0146] When the second encoder is an AI encoder, the method of encoding the current frame audio signal using the second encoder based on the bitrate allocated to the second encoder is similar to the method of encoding the first sub-signal using the AI ​​encoding module based on the bitrate allocated to the first sub-signal in step 303 above. When the second encoder is a non-AI encoder, the method of encoding the current frame audio signal using the second encoder based on the bitrate allocated to the second encoder is similar to the method of encoding the second sub-signal using a non-AI encoding module based on the bitrate allocated to the second sub-signal in step 303 above. Please refer to the description above for details; further elaboration will not be repeated here.

[0147] (3) Based on the code rate assigned to the first encoder, the first type of sub-signal is encoded using an AI encoding module and the second type of sub-signal is encoded using a non-AI encoding module.

[0148] In some embodiments, a bitrate is assigned to the plurality of sub-signals based on the bitrate assigned to the first encoder, such that the sum of the bitrates assigned to the plurality of sub-signals equals the bitrate assigned to the first encoder. Then, based on the bitrate assigned to the first type of sub-signals, an AI encoding module is used to encode the first type of sub-signals, and based on the bitrate assigned to the second type of sub-signals, a non-AI encoding module is used to encode the second type of sub-signals.

[0149] It should be noted that the implementation process of step (3) is similar to that of step 303 above, except that the sum of the code rates assigned to the multiple sub-signals is different.

[0150] In some embodiments, if the second encoder is an AI encoder, i.e., switching from an AI encoder to a hybrid encoder, and the current frame audio signal is encoded by both the AI ​​encoder and the hybrid encoder, then the bitstream corresponding to the current frame audio signal also includes: the frame state of the current frame audio signal, wherein the frame state indicates that the current frame audio signal is a switching frame from AI encoding to hybrid encoding.

[0151] When the second encoder is an AI encoder, the bitstream corresponding to the audio signal of the current frame also includes the bitrate assigned to the second encoder.

[0152] In other embodiments, if the second encoder is a non-AI encoder, i.e., switching from a non-AI encoder to a hybrid encoder, and the current frame audio signal is encoded by both the non-AI encoder and the hybrid encoder, then the bitstream corresponding to the current frame audio signal also includes: the frame state of the current frame audio signal, which indicates that the current frame audio signal is a switching frame from non-AI encoding to hybrid encoding.

[0153] When the encoder used to encode the current frame audio signal is different from that of the previous frame audio signal, i.e., the current frame audio signal is a switching frame, the bitstream corresponding to the current frame audio signal also includes the size of the encoding result of the current frame audio signal. The size of this encoding result includes the size of either the first encoded data or the second encoded data. In other words, carrying the size of either the first encoded data or the second encoded data in the bitstream facilitates the decoding end in parsing the first encoded data and the second encoded data from the payload portion of the bitstream.

[0154] It should be noted that the algorithm delays of the first encoder and the second encoder can be the same, so that the first encoder and the second encoder can encode the audio signal of the current frame in parallel. Of course, the algorithm delays of the first encoder and the second encoder can also be different, and this application does not limit this.

[0155] When the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is a hybrid encoder and the second encoder is a non-AI encoder, as shown in Figure 7, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-packet header, a first sub-payload, and a second sub-payload. The first sub-packet header is used to carry the sub-signal division information and sub-signal encoding information of the current frame audio signal, as well as the bit rate allocated to each sub-signal in the first type of sub-signal. The first sub-payload is used to carry the first encoded data, and the second sub-payload is used to carry the second encoded data.

[0156] Similarly, when the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is a hybrid encoder and the second encoder is an AI encoder, as shown in Figure 8, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-packet header, a first sub-payload, a second sub-packet header, and a second sub-payload. The first sub-packet header is used to carry the sub-signal division information and sub-signal encoding information of the current frame audio signal, as well as the bit rate allocated to each sub-signal in the first type of sub-signal. The first sub-payload is used to carry the first encoded data. The second sub-packet header is used to carry the bit rate allocated to the second encoder. The second sub-payload is used to carry the second encoded data.

[0157] It should be understood that the above is only an example. The information carried by the first sub-header and the second sub-header can also be carried entirely through the header. In this case, the payload includes the first sub-payload and the second sub-payload.

[0158] When the second encoder is a non-AI encoder and the payload includes a first sub-header, a first sub-payload, and a second sub-payload, the bitrate allocation process in step (1) above can be as follows: subtract the bitrate corresponding to the first sub-header from the bitrate corresponding to the payload to obtain the bitrates corresponding to the first sub-payload and the second sub-payload. Then, based on the bitrates corresponding to the first sub-payload and the second sub-payload, bitrates are allocated to the first encoder and the second encoder so that the sum of the bitrates allocated to the first encoder and the second encoder is equal to the bitrates corresponding to the first sub-payload and the second sub-payload.

[0159] When the second encoder is an AI encoder and the payload includes a first sub-header, a first sub-payload, a second sub-header, and a second sub-payload, the process of allocating the bitrate in step (1) above can be as follows: subtract the bitrates corresponding to the first and second sub-headers from the bitrate corresponding to the payload to obtain the bitrates corresponding to the first and second sub-payloads. Then, based on the bitrates corresponding to the first and second sub-payloads, bitrates are allocated to the first encoder and the second encoder so that the sum of the bitrates allocated to the first encoder and the second encoder is equal to the bitrates corresponding to the first and second sub-payloads.

[0160] The process of allocating code rates to the first encoder and the second encoder based on the code rates corresponding to the first and second sub-payloads is similar to the process of allocating code rates to the first encoder and the second encoder based on the code rates corresponding to the payload in step (1) above. The difference is that the sum of the code rates allocated to the first encoder and the code rates allocated to the second encoder is different.

[0161] The above describes the case where the first encoder is a hybrid encoder. In other embodiments, the first encoder can be an AI encoder or a non-AI encoder. These will be described separately below.

[0162] In the first scenario, the first encoder is an AI encoder. The current frame's audio signal is encoded using the AI ​​encoder to obtain the first encoded data.

[0163] Based on the first bit rate, the bit rate corresponding to the payload in the bit stream corresponding to the audio signal of the current frame is determined. Based on the bit rate corresponding to the payload, the audio signal of the current frame is encoded by an AI encoder to obtain the first encoded data.

[0164] The process of encoding the current frame's audio signal using an AI encoder based on the bitrate corresponding to the payload is similar to the method in step 303 above, where the first sub-signal is encoded using an AI encoding module based on the bitrate allocated to it. Please refer to the description above for details; it will not be repeated here.

[0165] In some embodiments, the bitstream corresponding to the current frame audio signal also includes the frame state of the current frame audio signal, which indicates that the current frame audio signal is an AI-encoded frame. That is, if the bitstream corresponding to the current frame audio signal includes data encoded by the AI ​​encoder but does not include data encoded by other encoders, then the current frame audio signal is determined to be an AI-encoded frame, i.e., the current frame audio signal is encoded using only the AI ​​encoder.

[0166] In some embodiments, when the current frame audio signal is encoded using an AI encoder, the bitstream corresponding to the current frame audio signal further includes the bitrate assigned to the first encoder. The bitrate assigned to the first encoder can be the bitrate corresponding to the payload.

[0167] It should be noted that the bitstream corresponding to the current frame audio signal includes a header and a payload. The header carries the frame state of the current frame audio signal and the bitrate allocated to the first encoder. The header also carries information such as the sampling rate, number of channels, and frame length. The payload carries the first encoded data.

[0168] In other embodiments, the plurality of encoders further includes a second encoder, which is the encoder used for the previous frame of audio signal, and the first encoder is different from the second encoder. Thus, the current frame of audio signal is encoded not only by the first encoder but also by the second encoder, where the first encoder is an AI encoder and the second encoder is not. In this case, a bitrate is allocated to the first encoder and the second encoder based on a first bitrate. Based on the bitrate allocated to the first encoder, the current frame of audio signal is encoded using the first encoder to obtain first encoded data. Based on the bitrate allocated to the second encoder, the current frame of audio signal is encoded using the second encoder to obtain second encoded data. The bitstream corresponding to the current frame of audio signal includes both the first encoded data and the second encoded data.

[0169] The implementation process of allocating code rates to the first encoder and the second encoder based on the first code rate is similar to the implementation process of allocating code rates to the first encoder and the second encoder based on the first code rate in step (1) above. For details, please refer to the description above.

[0170] The second encoder can be either a non-AI encoder or a hybrid encoder. The process of encoding the current frame's audio signal using the second encoder, based on the bitrate allocated to it, is similar to the encoding process described above for both non-AI and hybrid encoders. Please refer to the description above for details; it will not be repeated here.

[0171] In some embodiments, if the second encoder is a non-AI encoder (i.e., switching from a non-AI encoder to an AI encoder, where the current frame audio signal is encoded by both a non-AI encoder and an AI encoder), the bitstream corresponding to the current frame audio signal further includes: the frame state of the current frame audio signal, indicating that the current frame audio signal is a switching frame from non-AI encoding to AI encoding. If the second encoder is a hybrid encoder (i.e., switching from a hybrid encoder to an AI encoder, where the current frame audio signal is encoded by both a hybrid encoder and an AI encoder), the bitstream corresponding to the current frame audio signal further includes: the frame state of the current frame audio signal, indicating that the current frame audio signal is a switching frame from hybrid encoding to AI encoding.

[0172] When the current frame audio signal uses a different encoder than the previous frame audio signal, i.e., the current frame audio signal is a switching frame, the bitstream corresponding to the current frame audio signal also includes the size of the encoded result of the current frame audio signal. This encoded result includes the size of either the first encoded data or the second encoded data. In other words, carrying the size of the first encoded data or the second encoded data in the bitstream facilitates the decoding end in parsing the first encoded data and the second encoded data from the payload portion of the bitstream.

[0173] When the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is an AI encoder and the second encoder is a non-AI encoder, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-packet header, a first sub-payload, and a second sub-payload. The first sub-packet header is used to carry the bitrate allocated by the first encoder, the first sub-payload is used to carry the first encoded data, and the second sub-payload is used to carry the second encoded data.

[0174] Similarly, when the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is an AI encoder and the second encoder is a hybrid encoder, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-packet header, a first sub-payload, a second sub-packet header, and a second sub-payload. The first sub-packet header is used to carry the bitrate allocated by the first encoder, the first sub-payload is used to carry the first encoded data, the second sub-packet header is used to carry the sub-signal division information and sub-signal encoding information of the current frame audio signal, as well as the bitrate allocated to each sub-signal in the first type of sub-signal, and the second sub-payload is used to carry the second encoded data.

[0175] It should be understood that the above is only an example. The information carried by the first sub-header and the second sub-header can also be carried entirely through the header. In this case, the payload includes the first sub-payload and the second sub-payload.

[0176] When the second encoder is a non-AI encoder and the payload includes a first sub-header, a first sub-payload, and a second sub-payload, or when the second encoder is an AI encoder and the payload includes a first sub-header, a first sub-payload, a second sub-header, and a second sub-payload, the process of allocating bitrates to the first encoder and the second encoder is described above.

[0177] In the second scenario, the first encoder is a non-AI encoder. The current frame's audio signal is encoded using a non-AI encoder to obtain the first encoded data.

[0178] Based on the first bit rate, the bit rate corresponding to the payload in the bit stream corresponding to the audio signal of the current frame is determined. Based on the bit rate corresponding to the payload, the audio signal of the current frame is encoded using a non-AI encoder to obtain the first encoded data.

[0179] The process of encoding the current frame's audio signal using a non-AI encoder based on the bitrate corresponding to the payload is similar to the method in step 303 above, where a non-AI encoding module encodes the second sub-signal based on the bitrate allocated to the second sub-signal. Please refer to the description above for details; it will not be repeated here.

[0180] In some embodiments, the bitstream corresponding to the current frame audio signal also includes the frame state of the current frame audio signal, which indicates that the current frame audio signal is a non-AI encoded frame. That is, if the bitstream corresponding to the current frame audio signal includes data encoded by a non-AI encoder but does not include data encoded by other encoders, then the current frame audio signal is determined to be a non-AI encoded frame. In other words, the encoder used to encode the current frame audio signal is the same as that used to encode the previous frame audio signal, and both are non-AI encoders. In this case, the current frame audio signal is encoded using only a non-AI encoder.

[0181] In other embodiments, the plurality of encoders further includes a second encoder, which is the encoder used for the previous frame of audio signal, and the first encoder is different from the second encoder. Thus, the current frame audio signal is encoded not only by the first encoder but also by the second encoder, where the first encoder is a non-AI encoder and the second encoder is not a non-AI encoder. In this case, a bitrate is allocated to the first encoder and the second encoder based on a first bitrate. Based on the bitrate allocated to the first encoder, the current frame audio signal is encoded using the first encoder to obtain first encoded data. Based on the bitrate allocated to the second encoder, the current frame audio signal is encoded using the second encoder to obtain second encoded data. The bitstream corresponding to the current frame audio signal includes both the first encoded data and the second encoded data.

[0182] The implementation process of allocating code rates to the first encoder and the second encoder based on the first code rate is similar to the implementation process of allocating code rates to the first encoder and the second encoder based on the first code rate in step (1) above. For details, please refer to the description above.

[0183] The second encoder can be an AI encoder or a hybrid encoder. The process of encoding the current frame's audio signal using the second encoder, based on the bitrate assigned to it, is similar to the encoding process of the AI ​​encoder or hybrid encoder described above. Please refer to the description above for details; it will not be repeated here.

[0184] In some embodiments, if the second encoder is an AI encoder (i.e., switching from an AI encoder to a non-AI encoder, where the current frame audio signal is encoded by both AI and non-AI encoders), the bitstream corresponding to the current frame audio signal further includes: the frame state of the current frame audio signal, indicating that the current frame audio signal is a switching frame from AI encoding to non-AI encoding. If the second encoder is a hybrid encoder (i.e., switching from a hybrid encoder to a non-AI encoder, where the current frame audio signal is encoded by both hybrid encoder and non-AI encoders), the bitstream corresponding to the current frame audio signal further includes: the frame state of the current frame audio signal, indicating that the current frame audio signal is a switching frame from hybrid encoding to non-AI encoding.

[0185] When the current frame audio signal uses a different encoder than the previous frame audio signal, i.e., the current frame audio signal is a switching frame, the bitstream corresponding to the current frame audio signal also includes the size of the encoded result of the current frame audio signal. This encoded result includes the size of either the first encoded data or the second encoded data. In other words, carrying the size of the first encoded data or the second encoded data in the bitstream facilitates the decoding end in parsing the first encoded data and the second encoded data from the payload portion of the bitstream.

[0186] When the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is a non-AI encoder and the second encoder is an AI encoder, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-payload, a second sub-packet header, and a second sub-payload. The first sub-payload is used to carry the first encoded data, the second sub-packet header is used to carry the bitrate allocated to the second encoder, and the second sub-payload is used to carry the second encoded data.

[0187] Similarly, when the current frame audio signal is encoded simultaneously by the first encoder and the second encoder, and the first encoder is a non-AI encoder and the second encoder is a hybrid encoder, the packet header in the bitstream corresponding to the current frame audio signal is used to carry information such as the frame state of the current frame audio signal, the size of the encoding result of the current frame audio signal, the sampling rate, the number of channels, and the frame length. At this time, the payload in the bitstream corresponding to the current frame audio signal includes a first sub-payload, a second sub-packet header, and a second sub-payload. The first sub-payload is used to carry the first encoded data, the second sub-packet header is used to carry the sub-signal division information and sub-signal encoding information of the current frame audio signal, as well as the bit rate allocated to each sub-signal in the first type of sub-signal, and the second sub-payload is used to carry the second encoded data.

[0188] It should be understood that the above is only an example. The information carried by the first sub-header and the second sub-header can also be carried entirely through the header. In this case, the payload includes the first sub-payload and the second sub-payload.

[0189] When the second encoder is an AI encoder or a hybrid encoder, and the payload includes a first sub-payload, a second sub-header, and a second sub-payload, please refer to the above description for the process of allocating bit rates to the first encoder and the second encoder.

[0190] The following is a brief introduction to the above encoding content with reference to Figure 9. Referring to Figure 9, the multiple encoders include an AI encoder, a hybrid encoder, and a non-AI encoder. The encoding end obtains a first bitrate and determines the first encoder from the three encoders based on this first bitrate. The first bitrate is the encoding bitrate of the current frame audio signal, and the first encoder is used to encode the current frame audio signal. If the first encoder is an AI encoder, and the encoder of the previous frame audio signal is also an AI encoder (i.e., the second encoder is also an AI encoder), then the current frame audio signal is encoded using the AI ​​encoder to obtain the first encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the bitrate allocated to the first encoder, and the frame state of the current frame audio signal. This frame state indicates that the current frame audio signal is an AI encoded frame. If the first encoder is a hybrid encoder, and the encoder of the previous frame audio signal is also a hybrid encoder (meaning the second encoder is also a hybrid encoder), then the current frame audio signal is encoded using the hybrid encoder to obtain the first encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the bitrate allocated to each sub-signal in the first type of sub-signals, and the frame state of the current frame audio signal. This frame state indicates that the current frame audio signal is a hybrid encoded frame. If the first encoder is a non-AI encoder, and the encoder of the previous frame audio signal is also a non-AI encoder (meaning the second encoder is also a non-AI encoder), then the current frame audio signal is encoded using the non-AI encoder to obtain the first encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data and the frame state of the current frame audio signal. This frame state indicates that the current frame audio signal is a non-AI encoded frame.

[0191] If the first encoder is an AI encoder and the encoder of the previous frame audio signal is a hybrid encoder (i.e., the second encoder is a hybrid encoder), then the current frame audio signal is encoded by the AI ​​encoder to obtain first encoded data, and the current frame audio signal is encoded by the hybrid encoder to obtain second encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the bitrate allocated to the first encoder, the second encoded data, the size of the encoding result of the current frame audio signal, and the frame state of the current frame audio signal. The frame state indicates that the current frame audio signal is a switching frame from hybrid to AI. If the first encoder is an AI encoder and the encoder of the previous frame audio signal is a non-AI encoder (i.e., the second encoder is a non-AI encoder), then the current frame audio signal is encoded by the AI ​​encoder to obtain first encoded data, and the current frame audio signal is encoded by the non-AI encoder to obtain second encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the bitrate allocated to the first encoder, the second encoded data, the size of the encoding result of the current frame audio signal, and the frame state of the current frame audio signal. The frame state indicates that the current frame audio signal is a switching frame from non-AI to AI.

[0192] If the first encoder is a hybrid encoder and the encoder of the previous frame audio signal is an AI encoder, that is, the second encoder is an AI encoder, then the current frame audio signal is encoded by the hybrid encoder to obtain the first encoded data, and the current frame audio signal is encoded by the AI ​​encoder to obtain the second encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the bitrate allocated to each sub-signal in the first type of sub-signal, the second encoded data, the bitrate allocated to the second encoder, the size of the encoding result of the current frame audio signal, and the frame state of the current frame audio signal. The frame state indicates that the current frame audio signal is a switching frame from AI to hybrid. If the first encoder is a hybrid encoder and the encoder of the previous frame audio signal is a non-AI encoder, that is, the second encoder is a non-AI encoder, then the current frame audio signal is encoded by the hybrid encoder to obtain the first encoded data, and the current frame audio signal is encoded by the non-AI encoder to obtain the second encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the bitrate allocated to each sub-signal in the first type of sub-signal, the second encoded data, the size of the encoding result of the current frame audio signal, and the frame state of the current frame audio signal. The frame state indicates that the current frame audio signal is a switching frame from non-AI to hybrid.

[0193] If the first encoder is a non-AI encoder, and the encoder of the previous frame audio signal is an AI encoder (i.e., the second encoder is an AI encoder), then the current frame audio signal is encoded using the non-AI encoder to obtain first encoded data, and the current frame audio signal is encoded using the AI ​​encoder to obtain second encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the second encoded data, the bitrate allocated to the second encoder, the size of the encoding result of the current frame audio signal, and the frame state of the current frame audio signal. This frame state indicates that the current frame audio signal is a switching frame from AI to non-AI. If the first encoder is a non-AI encoder, and the encoder of the previous frame audio signal is a hybrid encoder (i.e., the second encoder is a hybrid encoder), then the current frame audio signal is encoded using the non-AI encoder to obtain first encoded data, and the current frame audio signal is encoded using the hybrid encoder to obtain second encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data, the second encoded data, the bitrate allocated to each sub-signal in the first type of sub-signal, the size of the encoding result of the current frame audio signal, and the frame state of the current frame audio signal. This frame state indicates that the current frame audio signal is a switching frame from hybrid to non-AI.

[0194] As can be seen, the frame state indicates three independent coded frame states: AI-coded frame, hybrid coded frame, and non-AI-coded frame, as well as six switching frame types: AI to non-AI, non-AI to AI, non-AI to hybrid, hybrid to non-AI, AI to hybrid, and hybrid to AI. Different frame states determine the data content of the payload in the bitstream corresponding to the current frame's audio signal.

[0195] It should be noted that, for frame switching, the above content refers to simultaneously encoding the audio signal of the current frame using both the first encoder and the second encoder. Of course, in some embodiments, the audio signal of the current frame can be directly encoded using the first encoder without using the second encoder, and this application does not limit this approach.

[0196] After the encoding end determines the bitstream corresponding to the current frame audio signal according to the above content, the bitstream corresponding to the current frame audio signal can be transmitted to the decoding end. One or more channels can be established between the encoding and decoding ends. If one channel is established between the encoding and decoding ends, the bitstream corresponding to the current frame audio signal can be directly transmitted to the decoding end through this channel. If multiple channels are established between the encoding and decoding ends, the bitstream corresponding to the current frame audio signal can be divided into multiple sub-bitstreams, and then transmitted to the decoding end through these multiple channels.

[0197] In some embodiments, high-importance sub-streams can be transmitted through high-priority channels, and low-importance sub-streams can be transmitted through low-priority channels. If all these channels are in good condition and the sub-streams transmitted through these channels are received by the decoding end, the decoding end can combine these sub-streams to obtain the bitstream corresponding to the current frame's audio signal, and then perform decoding as described below. However, if the low-priority channels are in poor condition, the sub-streams transmitted through high-priority channels can at least guarantee audio intelligibility.

[0198] In some embodiments, the plurality of sub-bitstreams includes a first sub-bitstream and a second sub-bitstream. The first sub-bitstream includes the bitstream of the low-frequency portion of the current frame audio signal, and the second sub-bitstream includes the bitstream of the high-frequency portion of the current frame audio signal; alternatively, the second sub-bitstream includes the bitstream of the entire current frame audio signal. The first sub-bitstream is of higher importance than the second sub-bitstream.

[0199] If the second sub-stream includes the entire audio signal of the current frame, it means that the first sub-stream is transmitted repeatedly. That is, the highly important sub-stream is transmitted redundantly. This not only ensures the stability of the connection, but also ensures the continuity of listening, and greatly improves the anti-interference capability of the channel.

[0200] It should be noted that for hybrid encoders, the payload portion of the bitstream corresponding to the audio signal of the current frame carries the encoded data of multiple sub-signals. Thus, when dividing the sub-bitstream, it can be done according to sub-signals, and the sub-signals in each sub-bitstream are consecutive. In this case, each sub-bitstream can also carry the number of sub-signals.

[0201] Figure 10 is a flowchart of an audio decoding method provided in an embodiment of this application. The method is applied to a decoding end, which can be the destination device in Figure 1 or the audio receiving device in Figure 2. Referring to Figure 10, the method includes steps 1001-1003.

[0202] Step 1001: Determine the first decoder from among multiple decoders to decode the audio signal of the current frame.

[0203] In some embodiments, the frame state of the current frame audio signal is obtained based on the bitstream corresponding to the current frame audio signal, and a first decoder is determined from a plurality of decoders based on the frame state of the current frame audio signal.

[0204] Based on the above description, the bitstream corresponding to the current frame audio signal includes the frame state of the current frame audio signal, and the frame state indicates the encoding status of the current frame audio signal. Therefore, the frame state of the current frame audio signal can be parsed from the bitstream corresponding to the current frame audio signal, and then, based on the frame state of the current frame audio signal, the first decoder can be determined from multiple decoders.

[0205] In some embodiments, the frame state indicates three independent encoded frame states: AI-encoded frame, hybrid-encoded frame, and non-AI-encoded frame, as well as six switching frame states: AI to non-AI, non-AI to AI, non-AI to hybrid, hybrid to non-AI, AI to hybrid, and hybrid to AI. Multiple decoders include AI decoders, hybrid decoders, and non-AI decoders. If the frame state of the current frame audio signal indicates that the current frame audio signal is an AI-encoded frame, then the AI ​​decoder among the multiple decoders is determined as the first decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a hybrid-encoded frame, then the hybrid decoder among the multiple decoders is determined as the first decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a non-AI-encoded frame, then the non-AI decoder among the multiple decoders is determined as the first decoder.

[0206] If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from AI to non-AI, then the non-AI decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the AI ​​decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from non-AI to AI, then the AI ​​decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the non-AI decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from non-AI to mixed, then the mixed decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the non-AI decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from mixed to non-AI, then the non-AI decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the mixed decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a transition frame from AI to mixed, then the mixed decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the AI ​​decoder. If the frame state of the current frame audio signal indicates that the current frame audio signal is a switching frame from mixing to AI, then the AI ​​decoder among the multiple decoders is determined as the first decoder, and the second decoder is determined as the mixing decoder.

[0207] Step 1002: If the first decoder is a hybrid decoder, then the hybrid decoder is used to parse the bitstream corresponding to the audio signal of the current frame to obtain multiple reconstructed sub-signals.

[0208] The hybrid decoder includes an AI decoding module and a non-AI decoding module. The reconstructed sub-signals include a first type of reconstructed sub-signals and a second type of reconstructed sub-signals. The first type of reconstructed sub-signals are obtained by parsing through the AI ​​decoding module, and the second type of reconstructed sub-signals are obtained by parsing through the non-AI decoding module.

[0209] In some embodiments, based on the bitstream corresponding to the current frame audio signal, sub-signal segmentation information and sub-signal encoding information are obtained. The sub-signal segmentation information indicates the segmentation method of multiple sub-signals, and the sub-signal encoding information indicates the encoding module corresponding to the multiple sub-signals. Based on the sub-signal segmentation information and sub-signal encoding information, a hybrid decoder is used to parse the bitstream corresponding to the current frame audio signal to obtain the reconstructed multiple sub-signals.

[0210] Since sub-signal segmentation information indicates the segmentation method of multiple sub-signals, and sub-signal encoding information indicates the encoding module corresponding to multiple sub-signals (i.e., whether each sub-signal is encoded using an AI encoding module or a non-AI encoding module), after obtaining the sub-signal segmentation information, the encoded data of each sub-signal can be distinguished from the bitstream corresponding to the current frame audio signal. Then, according to the encoding module corresponding to the multiple sub-signals, the bitstream corresponding to the current frame audio signal is parsed to obtain the reconstructed multiple sub-signals.

[0211] As an example, the encoding module corresponding to the first sub-signal is an AI encoding module. An AI decoding module is used to parse the encoded data of the first sub-signal in the bitstream corresponding to the current frame audio signal to obtain the reconstructed first sub-signal. The first sub-signal is any one of the multiple sub-signals corresponding to the AI ​​encoding module. The encoding module corresponding to the second sub-signal is a non-AI encoding module. A non-AI decoding module is used to parse the encoded data of the second sub-signal to obtain the reconstructed second sub-signal. The second sub-signal is any one of the multiple sub-signals corresponding to the non-AI encoding module.

[0212] AI decoding modules typically include one or more decoding networks, each supporting one or more bitrates. The process of using an AI decoding module to parse the encoded data of the first sub-signal in the bitstream corresponding to the current frame audio signal to obtain the reconstructed first sub-signal includes: obtaining the bitrate assigned to the first sub-signal based on the bitstream corresponding to the current frame audio signal; determining a first decoding network from among the one or more decoding networks based on the assigned bitrate, wherein the first decoding network supports a bitrate identical to the assigned bitrate of the first sub-signal; decoding and inverse quantizing the encoded data of the first sub-signal to obtain a reconstructed audio feature vector; and inputting the reconstructed audio feature vector into the first decoding network to obtain the reconstructed first sub-signal output by the first decoding network.

[0213] The AI ​​decoding module includes a decoding network that can be a deep neural network, which includes multiple residual convolutional layers. Of course, it can also be other types of networks, but this application embodiment does not limit this.

[0214] The following is a brief introduction to the AI ​​encoding and decoding process, combining the AI ​​encoding and decoding modules. Referring to Figures 11 and 12, the encoding end determines the first encoding network based on the bitrate allocated to the first sub-signal, inputs the first sub-signal into the first encoding network, and obtains the audio feature vector output by the first encoding network. This audio feature vector is quantized to obtain the quantization result, which is then encoded to obtain the encoded data of the first sub-signal. The encoding end transmits the encoded data of the first sub-signal to the decoding end. The decoding end determines the first decoding network based on the bitrate allocated to the first sub-signal. The encoded data of the first sub-signal is decoded and dequantized to obtain the reconstructed audio feature vector, which is then input into the first decoding network to obtain the reconstructed first sub-signal output by the first decoding network.

[0215] It should be noted that one or more encoding networks at the encoding end correspond one-to-one with one or more decoding networks at the decoding end.

[0216] The process of reconstructing the second sub-signal by parsing the encoded data of the second sub-signal using a non-AI decoding module includes: parsing and dequantizing the encoded data of the second sub-signal to obtain the dequantized spectrum; performing an inverse time-frequency transform on the dequantized spectrum to obtain time-frequency aliased data; adding a synthesis window to the time-frequency aliased data and overlapping it with the previous frame data to obtain the reconstructed second sub-signal.

[0217] The following is a brief introduction to the non-AI encoding and decoding process, using both the non-AI encoding and decoding modules. Referring to Figure 13, the encoding end applies an analysis window and performs time-frequency transformation on the second sub-signal to obtain its spectrum. Based on the assigned code rate, the spectrum is quantized and encoded to obtain the encoded data of the second sub-signal. The encoding end transmits the encoded data of the first sub-signal to the decoding end. The decoding end parses and dequantizes the encoded data of the second sub-signal to obtain the dequantized spectrum. An inverse time-frequency transformation is performed on the dequantized spectrum to obtain time-frequency aliased data. A synthesis window is applied to the time-frequency aliased data, and it is interleaved with the data from the previous frame to obtain the reconstructed second sub-signal.

[0218] The frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal, or the frequency corresponding to the first type of sub-signal is higher than the frequency corresponding to the second type of sub-signal.

[0219] Step 1003: Based on the reconstructed multiple sub-signals, obtain the reconstructed audio signal of the current frame.

[0220] In some embodiments, multiple reconstructed sub-signals are combined based on the division method indicated by the sub-signal division information to obtain the reconstructed current frame audio signal.

[0221] Steps 1002 and 1003 above describe the case where the current frame audio signal is a hybrid-coded frame, i.e., the current frame audio signal is decoded using a hybrid decoder alone. In other embodiments, the current frame audio signal may also be a switching frame from AI encoding to hybrid encoding, or a switching frame from non-AI encoding to hybrid encoding. In this case, the multiple decoders also include a second decoder, which can be an AI decoder or a non-AI decoder. The bitstream corresponding to the current frame audio signal includes first encoded data and second encoded data. The first encoded data is obtained by encoding the current frame audio signal using the first encoder, and the second encoded data is obtained by encoding the current frame audio signal using the second encoder. The reconstructed multiple sub-signals are obtained by parsing the first encoded data. Thus, the process of obtaining the reconstructed current frame audio signal based on the reconstructed multiple sub-signals includes the following steps.

[0222] (1) Based on the reconstructed sub-signals, determine the first reconstructed audio signal.

[0223] In some embodiments, multiple reconstructed sub-signals are combined based on the division method indicated by the sub-signal division information to obtain a first reconstructed audio signal.

[0224] (2) The second decoder is used to parse the bit stream corresponding to the audio signal of the current frame to obtain the second reconstructed audio signal. The second decoder is the decoder used for the audio signal of the previous frame.

[0225] In some embodiments, the size of the encoded result of the current frame audio signal is obtained based on the bitstream corresponding to the current frame audio signal. The size of the encoded result includes the size of the first encoded data or the second encoded data. Based on the size of the encoded result, a second decoder is used to parse the bitstream corresponding to the current frame audio signal to obtain the second reconstructed audio signal.

[0226] If the second decoder is an AI decoder, then the AI ​​decoder is used to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame to obtain the reconstructed second audio signal. The process of using the AI ​​decoder to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame is similar to the process of using the AI ​​decoding module to parse the encoded data of the first sub-signal in step 1002 above. For details, please refer to the description above; it will not be repeated here.

[0227] If the second decoder is a non-AI decoder, then the second encoded data in the bitstream corresponding to the current frame audio signal is parsed using the non-AI decoder to obtain the reconstructed second audio signal. The process of parsing the second encoded data in the bitstream corresponding to the current frame audio signal using a non-AI decoder is similar to the process of parsing the encoded data of the second sub-signal using a non-AI decoding module in step 1002 above. For details, please refer to the description above; it will not be repeated here.

[0228] (3) Based on the first reconstructed audio signal and the second reconstructed audio signal, the reconstructed current frame audio signal is obtained.

[0229] In some embodiments, an audio smoothing strategy is employed to smooth the first reconstructed audio signal and the second reconstructed audio signal to obtain the reconstructed current frame audio signal.

[0230] As an example, the end portion of the second reconstructed audio signal and the beginning portion of the first reconstructed audio signal are smoothed. The beginning portion of the second reconstructed audio signal is used as the beginning portion of the reconstructed current frame audio signal, and the smoothed portion is used as the end portion of the reconstructed current frame audio signal, thus obtaining the reconstructed current frame audio signal.

[0231] Taking the smoothing process of N sample points as an example, based on the data of N sample points at the end of the second reconstructed audio signal and the data of N sample points at the beginning of the first reconstructed audio signal, the smoothing process is performed according to the following formula (1).

[0232] In the above formula (1), Ci is the data of the i-th sample point after smoothing, Ai is the data of the i-th sample point of the end part of the second reconstructed audio signal, Bi is the data of the i-th sample point of the beginning part of the first reconstructed audio signal, C1 is the data of the first sample point after smoothing, A1 is the data of the first sample point of the end part of the second reconstructed audio signal, and B1 is the data of the first sample point of the beginning part of the first reconstructed audio signal.

[0233] As shown in Figure 14, the end portion of the second reconstructed audio signal, located between the two dashed lines, includes N sample data points, while the beginning portion of the first reconstructed audio signal, also located between the two dashed lines, includes N sample data points. In the above formula (1), 1-(i-1) / N represents the weight of the sample data points at the end of the second reconstructed audio signal, and (i-1) / N represents the weight of the sample data points at the beginning of the first reconstructed audio signal. As the value of i increases, the weight of the sample data points at the end of the second reconstructed audio signal decreases (the diagonal line at the end indicates the trend of weight change), while the weight of the sample data points at the beginning of the first reconstructed audio signal increases (the diagonal line at the beginning indicates the trend of weight change). This shows that the reconstructed audio signal of the current frame gradually transitions from the previous frame to the current frame, ensuring seamless switching between the audio data parsed by the two decoders.

[0234] It should be noted that the audio smoothing strategy shown above is only an example. In practical applications, other audio smoothing strategies can also be used to smooth the first reconstructed audio signal and the second reconstructed audio signal. This application embodiment does not limit this.

[0235] The switching from AI encoder to hybrid encoder is explained below with reference to Figure 15. Referring to Figure 15, the encoding end uses the AI ​​encoder to encode the first three frames of audio signal. If a bitrate increase is detected when encoding the fourth frame of audio signal, and a switch to the hybrid encoder is required, the encoding end uses the AI ​​encoder to encode the fourth frame of audio signal to obtain the second encoded data (the black-filled portion of the fourth frame), and uses the hybrid encoder to encode the fourth frame of audio signal to obtain the first encoded data (the white-filled portion of the fourth frame). At this time, the bitstream corresponding to the fourth frame of audio signal includes both the first and second encoded data. The decoding end then parses the first encoded data to obtain the first reconstructed audio signal, parses the second encoded data to obtain the second reconstructed audio signal, and smooths both the first and second reconstructed audio signals to obtain the reconstructed fourth frame of audio signal. The beginning portion (black-filled portion) of the reconstructed fourth frame of audio signal is the beginning portion of the second reconstructed audio signal, and the end portion (slash-filled portion) is the smoothed portion. The hybrid encoder is used for encoding starting from the fifth frame of audio signal.

[0236] The above describes the case where the first decoder is a hybrid decoder. In other embodiments, the first decoder can be an AI decoder or a non-AI decoder. These will be described separately below.

[0237] In the first scenario, the first decoder is an AI decoder. The AI ​​decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the reconstructed current frame audio signal.

[0238] The process of using an AI decoder to parse the first encoded data in the bitstream corresponding to the audio signal of the current frame is similar to the process of using an AI decoding module to parse the encoded data of the first sub-signal in step 1102 above. For details, please refer to the description above, which will not be repeated here.

[0239] In some embodiments, the current frame audio signal may also be a switching frame from non-AI encoding to AI encoding, or a switching frame from hybrid encoding to AI encoding. In this case, the multiple decoders also include a second decoder, which is either a non-AI decoder or a hybrid decoder. The bitstream corresponding to the current frame audio signal also includes second encoded data. Thus, the first encoded data in the bitstream corresponding to the current frame audio signal is parsed using the AI ​​decoder to obtain a first reconstructed audio signal, and the bitstream corresponding to the current frame audio signal is parsed using the second decoder to obtain a second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is obtained.

[0240] If the second decoder is a non-AI decoder, then the second encoded data in the bitstream corresponding to the current frame audio signal is parsed using the non-AI decoder to obtain the second reconstructed audio signal. The process of parsing the second encoded data in the bitstream corresponding to the current frame audio signal using a non-AI decoder is similar to the process of parsing the encoded data of the second sub-signal using a non-AI decoding module in step 1002 above. For details, please refer to the description above; it will not be repeated here.

[0241] If the second decoder is a hybrid decoder, then the hybrid decoder is used to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame to obtain the second reconstructed audio signal. The process of using a hybrid decoder to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame is similar to the process of using a hybrid decoder to parse the first encoded data in steps 1002 and 1003 above. For details, please refer to the description above; it will not be repeated here.

[0242] In the second scenario, the first decoder is a non-AI decoder. The non-AI decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the reconstructed current frame audio signal.

[0243] The process of using a non-AI decoder to parse the first encoded data in the bitstream corresponding to the audio signal of the current frame is similar to the process of using a non-AI decoding module to parse the encoded data of the second sub-signal in step 1002 above. For details, please refer to the description above, which will not be repeated here.

[0244] In some embodiments, the current frame audio signal may also be a switching frame from AI encoding to non-AI encoding, or a switching frame from hybrid encoding to non-AI encoding. In this case, the multiple decoders also include a second decoder, which is either an AI decoder or a hybrid decoder. The bitstream corresponding to the current frame audio signal also includes second encoded data. Thus, the first encoded data in the bitstream corresponding to the current frame audio signal is parsed using a non-AI decoder to obtain a first reconstructed audio signal, and the bitstream corresponding to the current frame audio signal is parsed using a second decoder to obtain a second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is obtained.

[0245] If the second decoder is an AI decoder, then the AI ​​decoder is used to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame to obtain the second reconstructed audio signal. The process of using the AI ​​decoder to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame is similar to the process of using the AI ​​decoding module to parse the encoded data of the first sub-signal in step 1002 above. For details, please refer to the description above; it will not be repeated here.

[0246] If the second decoder is a hybrid decoder, then the hybrid decoder is used to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame to obtain the second reconstructed audio signal. The process of using a hybrid decoder to parse the second encoded data in the bitstream corresponding to the audio signal of the current frame is similar to the process of using a hybrid decoder to parse the first encoded data in steps 1002 and 1003 above. For details, please refer to the description above; it will not be repeated here.

[0247] The decoding process described above is briefly explained below with reference to Figure 16. Referring to Figure 16, the multiple decoders include an AI decoder, a hybrid decoder, and a non-AI decoder. The decoding end obtains the frame state of the current frame audio signal from the bitstream corresponding to the current frame audio signal. If the frame state of the current frame audio signal indicates that the current frame audio signal is an AI-coded frame, the AI ​​decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the reconstructed current frame audio signal. If the frame state of the current frame audio signal indicates that the current frame audio signal is a hybrid-coded frame, the hybrid decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the reconstructed current frame audio signal. If the frame state of the current frame audio signal indicates that the current frame audio signal is a non-AI-coded frame, the non-AI decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the reconstructed current frame audio signal.

[0248] If the frame state of the current frame audio signal indicates that the current frame audio signal is a switching frame from AI to non-AI, then the non-AI decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the first reconstructed audio signal, and the AI ​​decoder is used to parse the second encoded data in the bitstream corresponding to the current frame audio signal to obtain the second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is determined.

[0249] If the frame state of the current frame audio signal indicates that the current frame audio signal is a non-AI to AI switching frame, then the AI ​​decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the first reconstructed audio signal, and the non-AI decoder is used to parse the second encoded data in the bitstream corresponding to the current frame audio signal to obtain the second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is determined.

[0250] If the frame state of the current frame audio signal indicates that the current frame audio signal is a non-AI to hybrid switching frame, then the hybrid decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the first reconstructed audio signal, and the non-AI decoder is used to parse the second encoded data in the bitstream corresponding to the current frame audio signal to obtain the second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is determined.

[0251] If the frame state of the current frame audio signal indicates that the current frame audio signal is a switching frame from hybrid to non-AI, then the non-AI decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the first reconstructed audio signal, and the hybrid decoder is used to parse the second encoded data in the bitstream corresponding to the current frame audio signal to obtain the second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is determined.

[0252] If the frame state of the current frame audio signal indicates that the current frame audio signal is a switching frame from AI to hybrid, then the hybrid decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the first reconstructed audio signal, and the AI ​​decoder is used to parse the second encoded data in the bitstream corresponding to the current frame audio signal to obtain the second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is determined.

[0253] If the frame state of the current frame audio signal indicates that the current frame audio signal is a switching frame from mixing to AI, then the AI ​​decoder is used to parse the first encoded data in the bitstream corresponding to the current frame audio signal to obtain the first reconstructed audio signal, and the mixing decoder is used to parse the second encoded data in the bitstream corresponding to the current frame audio signal to obtain the second reconstructed audio signal. Based on the first and second reconstructed audio signals, the reconstructed current frame audio signal is determined.

[0254] It can be seen that different decoders are used to reconstruct the audio signal of the current frame for different frame states.

[0255] In some cases, the bitstream corresponding to the current frame audio signal may also be lost. In this case, packet loss compensation can be performed based on the reconstructed previous frame or multiple frames of audio signal to obtain the reconstructed current frame audio signal. Alternatively, prediction can be performed based on the reconstructed previous frame audio signal and the reconstructed next frame audio signal to obtain the reconstructed current frame audio signal.

[0256] The above describes directly performing packet loss compensation on the reconstructed audio signal to obtain the reconstructed current frame audio signal. In other embodiments, packet loss compensation can also be performed using a decoder based on the frame state of the previous frame audio signal. That is, if the previous frame audio signal is an AI-coded frame, a frame switching from non-AI encoding to AI encoding, or a frame switching from hybrid encoding to AI encoding, packet loss compensation is performed using an AI decoder to obtain the reconstructed current frame audio signal. If the previous frame audio signal is a non-AI-coded frame, a frame switching from AI encoding to non-AI encoding, or a frame switching from hybrid encoding to non-AI encoding, packet loss compensation is performed using a non-AI decoder to obtain the reconstructed current frame audio signal. If the previous frame audio signal is a hybrid-coded frame, a frame switching from non-AI encoding to hybrid encoding, or a frame switching from AI encoding to hybrid encoding, packet loss compensation is performed using a hybrid decoder to obtain the reconstructed current frame audio signal.

[0257] In this embodiment, the AI ​​encoder, hybrid encoder, and non-AI encoder can seamlessly switch according to the bitrate, thereby adapting to different anti-interference environments, ensuring the continuity of audio signal transmission, and thus improving the audio playback effect. Furthermore, in the event of packet loss in the current frame, packet loss compensation can be performed on the audio signal of the current frame, thereby ensuring continuous and stutter-free audio signal transmission.

[0258] Figure 17 is a schematic diagram of an audio encoding device provided in an embodiment of this application. The device can be implemented as part or all of the encoding end by software, hardware, or a combination of both. The encoding end can be the source device in Figure 1 or the audio transmission device in Figure 2. Referring to Figure 17, the device includes: an encoder determination module 1701, a sub-signal acquisition module 1702, and multiple encoders 1703.

[0259] The encoder determination module 1701 is used to determine the first encoder 17031 from a plurality of encoders 1703 based on a first bit rate, wherein the first bit rate is the encoding bit rate of the audio signal of the current frame, and the first encoder 17031 is used to encode the audio signal of the current frame.

[0260] The sub-signal acquisition module 1702 is used to acquire multiple sub-signals based on the current frame audio signal if the first encoder 17031 is a hybrid encoder. The hybrid encoder includes an artificial intelligence (AI) encoding module and a non-AI encoding module.

[0261] The first encoder 17031 is used to encode a first type of sub-signal among multiple sub-signals using an AI encoding module and to encode a second type of sub-signal among multiple sub-signals using a non-AI encoding module to obtain first encoded data. The bitstream corresponding to the audio signal of the current frame includes the first encoded data.

[0262] In one possible implementation, the multiple sub-signals are multiple frequency band signals, which are obtained by dividing the spectrum of the current frame audio signal after time-frequency transformation; or, the multiple sub-signals are multiple sub-band signals, which are obtained by filtering the current frame audio signal.

[0263] In one possible implementation, the bitstream corresponding to the audio signal of the current frame also includes:

[0264] The current frame audio signal contains sub-signal segmentation information and sub-signal encoding information. The sub-signal segmentation information indicates the segmentation method of multiple sub-signals, and the sub-signal encoding information indicates the encoding module corresponding to multiple sub-signals.

[0265] In one possible implementation, the bitstream corresponding to the audio signal of the current frame also includes:

[0266] The frame state of the current frame audio signal indicates that the current frame audio signal is a mixed-coded frame.

[0267] In one possible implementation, the plurality of encoders 1703 further includes a second encoder, which is the encoder used for the previous frame of audio signal, and the device further includes:

[0268] A bitrate allocation module is used to allocate bitrates to the first encoder and the second encoder based on a first bitrate.

[0269] The second encoder is used to encode the current frame audio signal based on the bit rate allocated to the second encoder to obtain second encoded data. The bit stream corresponding to the current frame audio signal also includes the second encoded data.

[0270] The first encoder is specifically used for:

[0271] Based on the bit rate allocated to the first encoder, the first type of sub-signal is encoded using an AI encoding module and the second type of sub-signal is encoded using a non-AI encoding module.

[0272] In one possible implementation, the second encoder supports a lower bitrate than the first encoder, and the bitrate allocation module is specifically used for:

[0273] Assign a second bitrate to the second encoder, the second bitrate being the encoding bitrate of the previous frame of audio signal;

[0274] A third bitrate is assigned to the first encoder. The third bitrate is the difference between the bitrate corresponding to the payload in the bitstream and the second bitrate. The bitrate corresponding to the payload is determined based on the first bitrate.

[0275] In one possible implementation, the second encoder is an AI encoder, and the bitstream corresponding to the audio signal of the current frame also includes:

[0276] The frame state of the current frame audio signal indicates whether the current frame audio signal is a switching frame from AI encoding to hybrid encoding.

[0277] In one possible implementation, the second encoder supports a higher bitrate than the first encoder, and the bitrate allocation module is specifically used for:

[0278] Assign a fourth bitrate to the second encoder, which is the lowest bitrate supported by the second encoder;

[0279] A fifth bitrate is assigned to the first encoder. The fifth bitrate is the difference between the bitrate corresponding to the payload in the bitstream and the fourth bitrate. The bitrate corresponding to the payload is determined based on the first bitrate.

[0280] In one possible implementation, the second encoder is a non-AI encoder, and the bitstream corresponding to the audio signal of the current frame also includes:

[0281] The frame state of the current frame audio signal indicates that the current frame audio signal is a switching frame from non-AI encoding to hybrid encoding.

[0282] In one possible implementation, the bitstream corresponding to the audio signal of the current frame also includes the size of the encoding result of the audio signal of the current frame, and the size of the encoding result includes the size of the first encoded data or the second encoded data.

[0283] In one possible implementation, the frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal.

[0284] In this embodiment, the AI ​​encoding module has powerful compression capabilities (for the same data, the encoding bitrate required by the AI ​​encoding module is approximately 1 / 3 that of the non-AI encoding module, meaning that the AI ​​encoding effect at a bitrate of 6kbps is comparable to the non-AI encoding effect at a bitrate of around 15kbps). The bitrate required for encoding audio signals using the AI ​​encoding module is typically low, supporting bitrates below 8kbps, and even compressing narrowband audio signals to below 1kbps, but the compression complexity is usually high. The non-AI encoding module typically has lower compression complexity, but at low bitrates, using it to encode audio signals will reduce the audio encoding quality. Therefore, by combining the AI ​​encoding module and the non-AI encoding module, a portion of the multiple sub-signals is encoded by the AI ​​encoding module, and another portion by the non-AI encoding module. This leverages the powerful compression capabilities of the AI ​​encoding module to improve the audio encoding quality, while utilizing the low complexity of the non-AI encoding module to control computational complexity. In this way, the audio encoding quality can be improved while controlling the computational complexity, thus making up for the problem of poor audio quality at low bit rates in traditional audio encoding and decoding.

[0285] It should be noted that the audio encoding device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the audio encoding device and the audio encoding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0286] Figure 18 is a schematic diagram of an audio decoding device provided in an embodiment of this application. This device can be implemented as part or all of the decoding end by software, hardware, or a combination of both. The decoding end can be the destination device in Figure 1, or the audio receiving device in Figure 2. Referring to Figure 18, the device includes: a decoder determination module 1801, multiple decoders 1802, and an audio reconstruction module 1803.

[0287] Decoder determination module 1801 is used to determine, from a plurality of decoders 1802, a first decoder 18021 for decoding the audio signal of the current frame;

[0288] The first decoder 18021 is used to parse the bitstream corresponding to the audio signal of the current frame if the first decoder 18021 is a hybrid decoder, so as to obtain multiple reconstructed sub-signals;

[0289] The hybrid decoder includes an AI decoding module and a non-AI decoding module. The reconstructed sub-signals include reconstructed first-type sub-signals and reconstructed second-type sub-signals. The reconstructed first-type sub-signals are obtained by parsing through the AI ​​decoding module, and the reconstructed second-type sub-signals are obtained by parsing through the non-AI decoding module.

[0290] The audio reconstruction module 1803 is used to obtain the reconstructed audio signal of the current frame based on multiple reconstructed sub-signals.

[0291] In one possible implementation, the first decoder is specifically used for:

[0292] Based on the bitstream, sub-signal partitioning information and sub-signal encoding information are obtained. The sub-signal partitioning information is used to indicate the partitioning method of multiple sub-signals, and the sub-signal encoding information is used to indicate the encoding module corresponding to multiple sub-signals.

[0293] Based on the sub-signal segmentation information and sub-signal encoding information, a hybrid decoder is used to parse the bitstream to obtain the reconstructed multiple sub-signals.

[0294] In one possible implementation, the decoder determining module 1801 is specifically used for:

[0295] Based on the bitstream, the frame state of the current frame audio signal is obtained;

[0296] The first decoder is determined from multiple decoders based on the frame state.

[0297] In one possible implementation, the multiple decoders 1802 further include a second decoder, and the bitstream includes first encoded data and second encoded data. The first encoded data is obtained by encoding the current frame audio signal by the first encoder, and the second encoded data is obtained by encoding the current frame audio signal by the second encoder. The reconstructed multiple sub-signals are obtained by parsing the first encoded data.

[0298] Audio reconstruction module 1803 includes:

[0299] The first reconstruction submodule is used to determine the first reconstructed audio signal based on multiple reconstructed sub-signals;

[0300] The second decoder is used to parse the bitstream to obtain the second reconstructed audio signal. The second decoder is the decoder used for the previous frame of audio signal.

[0301] The second reconstruction submodule is used to obtain the reconstructed current frame audio signal based on the first and second reconstructed audio signals.

[0302] In one possible implementation, the second decoder is specifically used for:

[0303] Based on the bitstream, the size of the encoded result of the audio signal of the current frame is obtained. The size of the encoded result includes the size of the first encoded data or the second encoded data.

[0304] The bitstream is parsed based on the size of the encoding result to obtain the second reconstructed audio signal.

[0305] In one possible implementation, the frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal.

[0306] AI encoding modules offer powerful compression capabilities, but their compression complexity is typically high. Non-AI encoding modules, while generally less complex, suffer from reduced audio quality at low bitrates. Therefore, a combination of AI and non-AI encoding modules is employed. AI encoding modules encode a subset of the multiple sub-signals, while non-AI modules encode the remaining subset. This results in higher audio quality for the first type of sub-signals decoded by the AI ​​decoding module, and lower complexity for the second type. This approach improves audio quality while controlling computational complexity, addressing the issue of poor audio quality at low bitrates in traditional audio encoding and decoding.

[0307] It should be noted that the audio decoding device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the audio decoding device and the audio decoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0308] This application embodiment also provides an encoding device, the encoding device including: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the program or instructions are executed by the processor, the encoding device performs the above-described encoding method.

[0309] This application embodiment also provides a decoding device, the decoding device including: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the program or instructions are executed by the processor, the decoding device performs the above-described decoding method.

[0310] This application also provides an encoding / decoding system, which includes the encoding device and / or the decoding device.

[0311] This application also provides a computer-readable storage medium including program code, which, when run on a computer, causes the computer to perform the above-described encoding or decoding method.

[0312] This application also provides a computer program product, including instructions that, when executed on a computer, cause the computer to perform the above-described encoding or decoding method.

[0313] This application also provides a computer-readable storage medium storing a bitstream obtained according to the above-described encoding method.

[0314] This application also provides a device for storing a bitstream, including at least one storage medium and a communication interface; the communication interface is used to receive or send the bitstream; the at least one storage medium is used to store the bitstream; the bitstream is encoded by an encoder according to the above-described encoding method.

[0315] This application also provides a method for storing a bitstream, comprising: receiving a bitstream through a communication interface; storing the bitstream in one or more storage media, wherein the bitstream is encoded by an encoder according to the above-described encoding method.

[0316] This application also provides a system for distributing bitstreams, including at least one storage medium and a video streaming device; the at least one storage medium is used to store the bitstream, which is encoded by an encoder according to the above-described encoding method; the video streaming device is used to send the bitstream in the at least one storage medium to the decoder in response to a request from the decoder.

[0317] This application embodiment also provides a method for distributing a bitstream, comprising: receiving a first request; in response to the first request, selecting a bitstream from at least one storage medium; sending the bitstream to a destination device; wherein the at least one storage medium is used to store the bitstream, and the bitstream is encoded by an encoder according to the above-described encoding method.

[0318] This application also provides a system for processing bitstreams, including an image source device, an encoder, one or more storage media, and a destination device; the image source device is used to provide image data; the encoder is used to acquire the image data from the image source device through an interface and encode the image data to obtain one or more bitstreams, wherein the bitstreams are encoded by the encoder according to the above-described encoding method; the encoder is used to store the one or more bitstreams in one or more storage media; or the encoder is used to encapsulate the one or more bitstreams to obtain a transmission bitstream; the encoder is used to transmit the transmission bitstream to the destination device through a communication link or communication network; the destination device is used to decapsulate the transmission bitstream to obtain the one or more bitstreams; the destination device is used to decode the one or more bitstreams to obtain decoded data.

[0319] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0320] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0321] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. An audio encoding method, characterized in that, The method includes: A first encoder is determined from a plurality of encoders based on a first bitrate, wherein the first bitrate is the encoding bitrate of the current frame audio signal, and the first encoder is used to encode the current frame audio signal. If the first encoder is a hybrid encoder, then multiple sub-signals are obtained based on the current frame audio signal. The hybrid encoder includes an artificial intelligence (AI) encoding module and a non-AI encoding module. The AI ​​encoding module encodes the first type of sub-signal among the plurality of sub-signals, and the non-AI encoding module encodes the second type of sub-signal among the plurality of sub-signals to obtain first encoded data. The bitstream corresponding to the current frame audio signal includes the first encoded data.

2. The method as described in claim 1, characterized in that, The multiple sub-signals are multiple frequency band signals, which are obtained by dividing the spectrum of the current frame audio signal after time-frequency transformation; or, the multiple sub-signals are multiple sub-band signals, which are obtained by filtering the current frame audio signal.

3. The method as described in claim 1 or 2, characterized in that, The bitstream also includes: The sub-signal segmentation information and sub-signal encoding information of the current frame audio signal, wherein the sub-signal segmentation information is used to indicate the segmentation method of the multiple sub-signals, and the sub-signal encoding information is used to indicate the encoding module corresponding to the multiple sub-signals.

4. The method according to any one of claims 1-3, characterized in that, The bitstream also includes: The frame state of the current frame audio signal, wherein the frame state indicates that the current frame audio signal is a mixed-coded frame.

5. The method according to any one of claims 1-3, characterized in that, The plurality of encoders further includes a second encoder, which is the encoder used for the previous frame of audio signal, and the method further includes: Based on the first code rate, code rates are allocated to the first encoder and the second encoder; Based on the bitrate allocated to the second encoder, the current frame audio signal is encoded using the second encoder to obtain second encoded data, and the bitstream further includes the second encoded data; The step of encoding a first type of sub-signal among the plurality of sub-signals using the AI ​​encoding module and encoding a second type of sub-signal among the plurality of sub-signals using the non-AI encoding module includes: Based on the bitrate allocated to the first encoder, the first type of sub-signal is encoded using the AI ​​encoding module and the second type of sub-signal is encoded using the non-AI encoding module.

6. The method as described in claim 5, characterized in that, The second encoder supports a lower bitrate than the first encoder, and the step of allocating bitrates based on the first bitrate for the first encoder and the second encoder includes: Assign a second bitrate to the second encoder, the second bitrate being the encoding bitrate of the previous frame audio signal; A third bitrate is assigned to the first encoder. The third bitrate is the difference between the bitrate corresponding to the payload in the bitstream and the second bitrate. The bitrate corresponding to the payload is determined based on the first bitrate.

7. The method as described in claim 6, characterized in that, The second encoder is an AI encoder, and the bitstream further includes: The frame state of the current frame audio signal, wherein the frame state indicates that the current frame audio signal is a switching frame from AI encoding to hybrid encoding.

8. The method as described in claim 5, characterized in that, The second encoder supports a higher bitrate than the first encoder, and the step of allocating bitrates to the first encoder and the second encoder includes: Assign a fourth bitrate to the second encoder, the fourth bitrate being the lowest bitrate supported by the second encoder; A fifth bitrate is assigned to the first encoder. The fifth bitrate is the difference between the bitrate corresponding to the payload in the bitstream and the fourth bitrate. The bitrate corresponding to the payload is determined based on the first bitrate.

9. The method as described in claim 8, characterized in that, The second encoder is a non-AI encoder, and the bitstream further includes: The frame state of the current frame audio signal, wherein the frame state indicates that the current frame audio signal is a switching frame from non-AI encoding to hybrid encoding.

10. The method according to any one of claims 5-9, characterized in that, The bitstream also includes the size of the encoding result of the current frame audio signal, and the size of the encoding result includes the size of the first encoded data or the second encoded data.

11. The method according to any one of claims 1-10, characterized in that, The frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal.

12. An audio decoding method, characterized in that, The method includes: Determine the first decoder from among multiple decoders to decode the audio signal of the current frame; If the first decoder is a hybrid decoder, then the hybrid decoder is used to parse the bitstream corresponding to the audio signal of the current frame to obtain multiple reconstructed sub-signals; The hybrid decoder includes an AI decoding module and a non-AI decoding module. The reconstructed multiple sub-signals include reconstructed first-type sub-signals and reconstructed second-type sub-signals. The reconstructed first-type sub-signals are obtained by parsing through the AI ​​decoding module, and the reconstructed second-type sub-signals are obtained by parsing through the non-AI decoding module. Based on the reconstructed multiple sub-signals, the reconstructed current frame audio signal is obtained.

13. The method as described in claim 12, characterized in that, The process of using the hybrid decoder to parse the bitstream corresponding to the current frame audio signal to obtain multiple reconstructed sub-signals includes: Based on the bitstream, sub-signal partitioning information and sub-signal encoding information are obtained. The sub-signal partitioning information is used to indicate the partitioning method of the multiple sub-signals, and the sub-signal encoding information is used to indicate the encoding module corresponding to the multiple sub-signals. Based on the sub-signal segmentation information and the sub-signal encoding information, the hybrid decoder is used to parse the bitstream to obtain the reconstructed multiple sub-signals.

14. The method as described in claim 12 or 13, characterized in that, Determining the first decoder from a plurality of decoders includes: Based on the bitstream, the frame state of the current frame audio signal is obtained; Based on the frame state, the first decoder is determined from the plurality of decoders.

15. The method according to any one of claims 12-14, characterized in that, The plurality of decoders further includes a second decoder, the bitstream includes first encoded data and second encoded data, the first encoded data is obtained by encoding the current frame audio signal by the first encoder, the second encoded data is obtained by encoding the current frame audio signal by the second encoder, and the reconstructed plurality of sub-signals are obtained by parsing the first encoded data; The reconstructed current frame audio signal, obtained based on the reconstructed multiple sub-signals, includes: Based on the reconstructed multiple sub-signals, a first reconstructed audio signal is determined; The second decoder is used to parse the bitstream to obtain the second reconstructed audio signal. The second decoder is the decoder used in the previous frame of audio signal. Based on the first reconstructed audio signal and the second reconstructed audio signal, the reconstructed current frame audio signal is obtained.

16. The method as described in claim 15, characterized in that, The step of using the second decoder to parse the bitstream to obtain the second reconstructed audio signal includes: Based on the bitstream, the size of the encoding result of the current frame audio signal is obtained, and the size of the encoding result includes the size of the first encoded data or the second encoded data; Based on the size of the encoding result, the second decoder is used to parse the bitstream to obtain the second reconstructed audio signal.

17. The method according to any one of claims 12-16, characterized in that, The frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal.

18. An audio encoding device, characterized in that, The device includes: An encoder determination module is used to determine a first encoder from a plurality of encoders based on a first bit rate, wherein the first bit rate is the encoding bit rate of the current frame audio signal, and the first encoder is used to encode the current frame audio signal; The sub-signal acquisition module is used to acquire multiple sub-signals based on the current frame audio signal if the first encoder is a hybrid encoder. The hybrid encoder includes an artificial intelligence (AI) encoding module and a non-AI encoding module. The first encoder is used to encode a first type of sub-signal among the plurality of sub-signals using the AI ​​encoding module and to encode a second type of sub-signal among the plurality of sub-signals using the non-AI encoding module to obtain first encoded data, wherein the bitstream corresponding to the current frame audio signal includes the first encoded data.

19. The apparatus as claimed in claim 18, characterized in that, The plurality of encoders further includes a second encoder, which is the encoder used for the previous frame of audio signal, and the device further includes: The bitrate allocation module is used to allocate bitrates to the first encoder and the second encoder based on the first bitrate; The second encoder is used to encode the current frame audio signal based on the bitrate allocated to the second encoder to obtain second encoded data, and the bitstream further includes the second encoded data; The first encoder is specifically used for: Based on the bitrate allocated to the first encoder, the first type of sub-signal is encoded using the AI ​​encoding module and the second type of sub-signal is encoded using the non-AI encoding module.

20. The apparatus as claimed in claim 19, characterized in that, The second encoder supports a lower bitrate than the first encoder, and the bitrate allocation module is specifically used for: Assign a second bitrate to the second encoder, the second bitrate being the encoding bitrate of the previous frame audio signal; A third bitrate is assigned to the first encoder. The third bitrate is the difference between the bitrate corresponding to the payload in the bitstream and the second bitrate. The bitrate corresponding to the payload is determined based on the first bitrate.

21. The apparatus as claimed in claim 19, characterized in that, The second encoder supports a higher bitrate than the first encoder, and the bitrate allocation module is specifically used for: Assign a fourth bitrate to the second encoder, the fourth bitrate being the lowest bitrate supported by the second encoder; A fifth bitrate is assigned to the first encoder. The fifth bitrate is the difference between the bitrate corresponding to the payload in the bitstream and the fourth bitrate. The bitrate corresponding to the payload is determined based on the first bitrate.

22. The apparatus according to any one of claims 18-21, characterized in that, The frequency corresponding to the first type of sub-signal is lower than the frequency corresponding to the second type of sub-signal.

23. An audio decoding device, characterized in that, The device includes: A decoder determination module is used to determine, from a plurality of decoders, the first decoder used to decode the audio signal of the current frame; The first decoder is configured to, if the first decoder is a hybrid decoder, parse the bitstream corresponding to the audio signal of the current frame to obtain multiple reconstructed sub-signals; The hybrid decoder includes an AI decoding module and a non-AI decoding module. The reconstructed multiple sub-signals include reconstructed first-type sub-signals and reconstructed second-type sub-signals. The reconstructed first-type sub-signals are obtained by parsing through the AI ​​decoding module, and the reconstructed second-type sub-signals are obtained by parsing through the non-AI decoding module. An audio reconstruction module is used to obtain the reconstructed audio signal of the current frame based on the reconstructed multiple sub-signals.

24. The apparatus as claimed in claim 23, characterized in that, The decoder determination module is specifically used for: Based on the bitstream, the frame state of the current frame audio signal is obtained; Based on the frame state, the first decoder is determined from the plurality of decoders.

25. The apparatus as claimed in claim 23 or 24, characterized in that, The plurality of decoders further includes a second decoder, the bitstream includes first encoded data and second encoded data, the first encoded data is obtained by encoding the current frame audio signal by the first encoder, the second encoded data is obtained by encoding the current frame audio signal by the second encoder, and the reconstructed plurality of sub-signals are obtained by parsing the first encoded data; The audio reconstruction module includes: The first reconstruction submodule is used to determine the first reconstructed audio signal based on the multiple reconstructed sub-signals; The second decoder is used to parse the bitstream to obtain the second reconstructed audio signal. The second decoder is the decoder used in the previous frame of audio signal. The second reconstruction submodule is used to obtain the reconstructed current frame audio signal based on the first reconstructed audio signal and the second reconstructed audio signal.

26. An encoding device, characterized in that, The encoding device includes: a processor coupled to a memory for storing programs or instructions, which, when executed by the processor, cause the encoding device to perform the method as described in any one of claims 1-11.

27. A decoding device, characterized in that, The decoding device includes: a processor coupled to a memory for storing programs or instructions, which, when executed by the processor, cause the decoding device to perform the method as described in any one of claims 12-17.

28. A codec system, characterized in that, The encoding / decoding system includes the encoding device as described in claim 26, and / or the decoding device as described in claim 27.

29. A computer-readable storage medium, characterized in that, Includes program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-11 or the method as described in any one of claims 12-17.

30. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-11 or the method as described in any one of claims 12-17.