An audio signal encoding and decoding method and apparatus

Abstract

This application discloses an audio signal encoding and decoding method and apparatus to improve encoding quality and audio signal reconstruction effect. The audio signal encoding method includes: obtaining M transient identifiers for M blocks based on the spectra of M blocks in the current frame of the audio signal to be encoded; the M blocks include a first block, where the transient identifiers of the first block indicate whether the first block is a transient block or a non-transient block; obtaining grouping information for the M blocks based on the M transient identifiers; grouping and arranging the spectra of the M blocks according to the grouping information to obtain the spectrum to be encoded in the current frame; encoding the spectrum to be encoded using an encoding neural network to obtain a spectrum encoding result; and writing the spectrum encoding result into a bitstream.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio signal encoding and decoding method and apparatus. Background Technology

[0002] Audio data compression is an indispensable part of media applications such as media communication and media broadcasting. With the development of the high-definition audio industry and the 3D audio industry, people's demand for audio quality is getting higher and higher, resulting in a rapid increase in the amount of audio data in media applications.

[0003] Current audio data compression technology is based on the basic principles of signal processing. It uses the correlation of signals in time and space to compress the original audio signal to reduce the amount of data, thereby facilitating the transmission or storage of audio data.

[0004] Current audio signal encoding schemes suffer from low encoding quality when the audio signal is transient. Poor audio signal reconstruction also occurs during signal reconstruction at the decoding end. Summary of the Invention

[0005] This application provides an audio signal encoding and decoding method and apparatus to improve encoding quality and audio signal reconstruction effect.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions:

[0007] In a first aspect, embodiments of this application provide an audio signal encoding method, comprising: obtaining M transient identifiers of the M blocks based on the spectra of M blocks of the current frame of the audio signal to be encoded; the M blocks including a first block, wherein the transient identifiers of the first block are used to indicate that the first block is a transient block or to indicate that the first block is a non-transient block; obtaining grouping information of the M blocks based on the M transient identifiers; grouping and arranging the spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum to be encoded of the current frame; encoding the spectrum to be encoded using an encoding neural network to obtain a spectrum encoding result; and writing the spectrum encoding result into a bitstream.

[0008] In the above scheme, M transient identifiers of M blocks are obtained based on the spectra of M blocks in the current frame of the audio signal to be encoded. After obtaining the grouping information of the M blocks based on the M transient identifiers, the grouping information of the M blocks can be used to group and arrange the spectra of the M blocks in the current frame. By grouping and arranging the spectra of the M blocks, the arrangement order of the spectra of the M blocks in the current frame can be adjusted. After obtaining the spectrum to be encoded in the current frame, the spectrum to be encoded is encoded using an encoding neural network to obtain the spectrum encoding result, which can be carried in the bitstream. Therefore, in this embodiment, the spectra of M blocks can be grouped and arranged according to the M transient identifiers in the current frame of the audio signal, thereby realizing the grouping, arrangement, and encoding of blocks with different transient identifiers, improving the encoding quality of the audio signal.

[0009] In one possible implementation, the method further includes: encoding the grouping information of the M blocks to obtain a grouping information encoding result; and writing the grouping information encoding result into the bitstream. In the above scheme, after obtaining the grouping information of the M blocks, the encoding end can carry the grouping information in the bitstream. First, the grouping information is encoded; the encoding method used for the grouping information is not limited here. By encoding the grouping information, a grouping information encoding result can be obtained, which can be written into the bitstream, thereby enabling the bitstream to carry the grouping information encoding result.

[0010] In one possible implementation, the grouping information of the M blocks includes: the number of groups or a group number identifier of the M blocks, wherein the group number identifier indicates the number of groups; when the number of groups is greater than 1, the grouping information of the M blocks further includes: M transient identifiers of the M blocks; or, the grouping information of the M blocks includes: M transient identifiers of the M blocks. In the above scheme, the grouping information of the M blocks includes: the number of groups or a group number identifier of the M blocks, wherein the group number identifier indicates the number of groups; when the number of groups is greater than 1, the grouping information of the M blocks further includes: M transient identifiers of the M blocks; or, the grouping information of the M blocks includes: M transient identifiers of the M blocks. The grouping information of the M blocks can indicate the grouping status of the M blocks, so that the encoding end can use this grouping information to group and arrange the spectrum of the M blocks.

[0011] In one possible implementation, grouping and arranging the spectra of the M blocks according to their grouping information to obtain the spectrum to be encoded for the current frame includes: assigning the spectra of the M blocks that are indicated as transient blocks by the M transient identifiers to a transient group, and assigning the spectra of the M blocks that are indicated as non-transient blocks by the M transient identifiers to a non-transient group; and arranging the spectra of the blocks in the transient group before the spectra of the blocks in the non-transient group to obtain the spectrum to be encoded for the current frame. In the above scheme, after the encoding end obtains the grouping information of the M blocks, it groups the M blocks based on the different transient identifiers to obtain transient and non-transient groups. Next, the positions of the M blocks in the spectrum of the current frame are arranged, with the spectra of the blocks in the transient group arranged before the spectra of the blocks in the non-transient group to obtain the spectrum to be encoded. That is, in the spectrum to be encoded, the spectrum of all transient blocks is located before the spectrum of non-transient blocks, so that the spectrum of transient blocks can be adjusted to a position with higher coding importance, so that the audio signal reconstructed after processing by neural network encoding and decoding can better retain transient features.

[0012] In one possible implementation, grouping and arranging the spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum to be encoded for the current frame includes: arranging the spectra of the M blocks that are indicated as transient blocks by the M transient identifiers before the spectra of the M blocks that are indicated as non-transient blocks by the M transient identifiers, thereby obtaining the spectrum to be encoded for the current frame. In the above scheme, after the encoding end obtains the grouping information of the M blocks, it determines the transient identifier of each block in the M blocks according to the grouping information. First, it finds P transient blocks and Q non-transient blocks from the M blocks, then M = P + Q. Arranging the spectra of the M blocks that are indicated as transient blocks by the M transient identifiers before the spectra of the M blocks that are indicated as non-transient blocks by the M transient identifiers, thereby obtaining the spectrum to be encoded for the current frame. That is, in the spectrum to be encoded, the spectrum of all transient blocks is located before the spectrum of non-transient blocks, so that the spectrum of transient blocks can be adjusted to a position with higher coding importance, so that the audio signal reconstructed after processing by neural network encoding and decoding can better retain transient features.

[0013] In one possible implementation, before encoding the spectrum to be encoded using a coding neural network, the method further includes: performing intra-group interleaving on the spectrum to be encoded to obtain the spectra of M interleaved blocks; the encoding of the spectrum to be encoded using a coding neural network includes: encoding the spectra of the M interleaved blocks using a coding neural network. In the above scheme, after obtaining the spectrum to be encoded in the current frame, the encoding end can first perform intra-group interleaving according to the grouping of M blocks to obtain the spectra of the M interleaved blocks. The spectra of the M interleaved blocks can then be the input data of the coding neural network. Intra-group interleaving can also reduce the side information in the encoding and improve encoding efficiency.

[0014] In one possible implementation, the number of blocks identified as transient by the M transient flags is P, and the number of blocks identified as non-transient by the M transient flags is Q, where M = P + Q. The intra-group interleaving process for the spectrum to be encoded includes: interleaving the spectra of the P blocks to obtain interleaved spectra of the P blocks; interleaving the spectra of the Q blocks to obtain interleaved spectra of the Q blocks; and encoding the intra-group interleaved spectra of the M blocks using a coding neural network includes: encoding the interleaved spectra of the P blocks and the interleaved spectra of the Q blocks using a coding neural network. In the above scheme, interleaving the spectra of the P blocks includes treating the spectra of the P blocks as a whole for interleaving; similarly, interleaving the spectra of the Q blocks includes treating the spectra of the Q blocks as a whole for interleaving. The encoder can perform interleaving processing separately for transient and non-transient groups, thereby obtaining the spectra of P interleaved blocks and Q interleaved blocks. The spectra of the P interleaved blocks and the spectra of the Q interleaved blocks can be used as input data for the coding neural network. Intra-group interleaving can also reduce side information in the encoding, improving coding efficiency.

[0015] In one possible implementation, before obtaining the M transient identifiers of the M blocks based on the spectra of the M blocks of the current frame of the audio signal to be encoded, the method further includes: obtaining the window type of the current frame, wherein the window type is a short window type or a non-short window type; the step of obtaining the M transient identifiers of the M blocks based on the spectra of the M blocks of the current frame of the audio signal to be encoded is only performed when the window type is a short window type. In the above scheme, in this embodiment of the application, the aforementioned encoding scheme can be executed only when the window type of the current frame is a short window type, thereby realizing encoding when the audio signal is a transient signal.

[0016] In one possible implementation, the method further includes: encoding the window type to obtain a window type encoding result; and writing the window type encoding result into the bitstream. In the above scheme, after obtaining the window type of the current frame, the encoding end can carry the window type in the bitstream. First, the window type is encoded; the encoding method used for the window type is not limited here. By encoding the window type, a window type encoding result can be obtained, which can be written into the bitstream, thereby enabling the bitstream to carry the window type encoding result.

[0017] In one possible implementation, obtaining the M transient identifiers of the M blocks based on the spectra of the M blocks of the current frame of the audio signal to be encoded includes: obtaining the M spectral energies of the M blocks based on the spectra of the M blocks; obtaining the average spectral energy of the M blocks based on the M spectral energies; and obtaining the M transient identifiers of the M blocks based on the M spectral energies and the average spectral energy. In the above scheme, after the encoding end obtains the M spectral energies, it can average the M spectral energies to obtain the average spectral energy, or remove the maximum value or several of the largest values ​​from the M spectral energies before averaging to obtain the average spectral energy. By comparing the spectral energy of each block in the M spectral energies with the average spectral energy, the change in the spectrum of each block compared to the spectra of other blocks in the M blocks is determined, thereby obtaining the M transient identifiers of the M blocks, wherein the transient identifier of a block can be used to represent the transient characteristics of a block. In this embodiment, the transient identifier of each block can be determined by the spectral energy of each block and the average spectral energy, so that the transient identifier of a block can determine the grouping information of the block.

[0018] In one possible implementation, when the spectral energy of the first block is greater than K times the average spectral energy, the transient flag of the first block indicates that the first block is a transient block; or, when the spectral energy of the first block is less than or equal to K times the average spectral energy, the transient flag of the first block indicates that the first block is a non-transient block; wherein, K is a real number greater than or equal to 1. Taking the determination of the transient flag of the first block out of M blocks as an example, in the above scheme, when the spectral energy of the first block is greater than K times the average spectral energy, it indicates that the spectrum of the first block changes too much compared to the other M blocks, and in this case, the transient flag of the first block indicates that the first block is a transient block. When the spectral energy of the first block is less than or equal to K times the average spectral energy, it indicates that the spectrum of the first block does not change much compared to the other M blocks, and the transient flag of the first block indicates that the first block is a non-transient block.

[0019] Secondly, embodiments of this application also provide an audio signal decoding method, comprising: obtaining grouping information of M blocks of the current frame of the audio signal from a bitstream, wherein the grouping information is used to indicate M transient identifiers of the M blocks; decoding the bitstream using a decoding neural network to obtain the decoded spectrum of the M blocks; performing inverse grouping and arrangement processing on the decoded spectrum of the M blocks according to the grouping information of the M blocks to obtain the spectrum of the inverse grouping and arrangement processing of the M blocks; and obtaining the reconstructed audio signal of the current frame according to the spectrum of the inverse grouping and arrangement processing of the M blocks.

[0020] In the above scheme, the grouping information of M blocks of the current frame of the audio signal is obtained from the bitstream. This grouping information indicates the M transient identifiers of the M blocks. The bitstream is decoded using a decoding neural network to obtain the decoded spectrum of the M blocks. Based on the grouping information of the M blocks, the decoded spectrum of the M blocks is reverse-grouped to obtain the spectrum of the reverse-grouped M blocks. The reconstructed audio signal of the current frame is then obtained from the spectrum of the reverse-grouped M blocks. Since the spectral encoding result included in the bitstream is grouped, decoding the bitstream yields the decoded spectrum of the M blocks. Then, through reverse grouping, the spectrum of the reverse-grouped M blocks can be obtained, thus yielding the reconstructed audio signal of the current frame. During signal reconstruction, reverse grouping and decoding can be performed based on blocks with different transient identifiers in the audio signal, thereby improving the audio signal reconstruction effect.

[0021] In one possible implementation, before performing the reverse grouping and rearranging process on the decoded spectra of the M blocks according to the grouping information of the M blocks, the method further includes: performing intra-group deinterleaving processing on the decoded spectra of the M blocks to obtain the intra-group deinterleaving processed spectra of the M blocks; the reverse grouping and rearranging process on the decoded spectra of the M blocks according to the grouping information of the M blocks includes: performing the reverse grouping and rearranging process on the intra-group deinterleaving processed spectra of the M blocks according to the grouping information of the M blocks.

[0022] In one possible implementation, the number of blocks identified as transient by the M transient identifiers is P, and the number of blocks identified as non-transient by the M transient identifiers is Q, where M = P + Q; the intra-group deinterleaving process for the decoded spectra of the M blocks includes: deinterleaving the decoded spectra of the P blocks; and deinterleaving the decoded spectra of the Q blocks.

[0023] In one possible implementation, the number of blocks identified as transient by the M transient identifiers is P, and the number of blocks identified as non-transient by the M transient identifiers is Q, where M = P + Q. The step of performing the reverse grouping arrangement process on the decoded spectrum of the M blocks based on the grouping information of the M blocks includes: obtaining the indices of the P blocks based on the grouping information of the M blocks; obtaining the indices of the Q blocks based on the grouping information of the M blocks; and performing the reverse grouping arrangement process on the decoded spectrum of the M blocks based on the indices of the P blocks and the indices of the Q blocks.

[0024] In one possible implementation, the method further includes: obtaining the window type of the current frame from the bitstream, wherein the window type is a short window type or a non-short window type; and performing the step of obtaining the grouping information of the M blocks of the current frame from the bitstream only when the window type of the current frame is a short window type.

[0025] In one possible implementation, the grouping information of the M blocks includes: the number of groups or a group number identifier of the M blocks, wherein the group number identifier is used to indicate the number of groups; when the number of groups is greater than 1, the grouping information of the M blocks further includes: M transient identifiers of the M blocks; or, the grouping information of the M blocks includes: M transient identifiers of the M blocks.

[0026] Thirdly, embodiments of this application also provide an audio signal encoding device, comprising:

[0027] A transient identifier acquisition module is used to obtain M transient identifiers of the M blocks based on the spectrum of the M blocks of the current frame of the audio signal to be encoded; the M blocks include a first block, and the transient identifier of the first block is used to indicate that the first block is a transient block or to indicate that the first block is a non-transient block;

[0028] The grouping information acquisition module is used to obtain the grouping information of the M blocks based on the M transient identifiers of the M blocks;

[0029] The grouping and arranging module is used to group and arrange the spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum to be encoded;

[0030] The encoding module is used to encode the spectrum to be encoded using an encoding neural network to obtain a spectrum encoding result; and to write the spectrum encoding result into a bitstream.

[0031] In a third aspect of this application, the constituent modules of the audio signal encoding device may also perform the steps described in the first aspect and various possible implementations, as detailed in the foregoing description of the first aspect and various possible implementations.

[0032] Fourthly, embodiments of this application also provide an audio signal decoding device, comprising:

[0033] A grouping information acquisition module is used to obtain grouping information of M blocks of the current frame of the audio signal from the bitstream, wherein the grouping information is used to indicate M transient identifiers of the M blocks;

[0034] A decoding module is used to decode the bitstream using a decoding neural network to obtain the decoding spectrum of M blocks;

[0035] The reverse grouping and arrangement module is used to perform reverse grouping and arrangement processing on the decoded spectrum of the M blocks according to the grouping information of the M blocks, so as to obtain the spectrum of the reverse grouping and arrangement processing of the M blocks;

[0036] An audio signal acquisition module is used to obtain a reconstructed audio signal based on the spectrum of the M blocks in reverse grouping and arrangement processing.

[0037] In the fourth aspect of this application, the constituent modules of the audio signal decoding device may also perform the steps described in the first aspect and various possible implementations, as detailed in the foregoing description of the first aspect and various possible implementations.

[0038] Fifthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first or second aspect above.

[0039] In a sixth aspect, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods described in the first or second aspect above.

[0040] In a seventh aspect, embodiments of this application provide a computer-readable storage medium including a bitstream generated by the method described in the first aspect above.

[0041] Eighthly, embodiments of this application provide a communication device, which may include entities such as terminal devices or chips. The communication device includes: a processor and a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, causing the communication device to perform the method as described in any one of the first or second aspects above.

[0042] Ninthly, this application provides a chip system including a processor for supporting an audio encoder or audio decoder in implementing the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing necessary program instructions and data for the audio encoder or audio decoder. This chip system may be composed of chips or may include chips and other discrete devices.

[0043] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0044] In one embodiment of this application, M transient identifiers of M blocks are obtained based on the spectra of M blocks in the current frame of the audio signal to be encoded. After obtaining the grouping information of the M blocks based on the M transient identifiers, the grouping information of the M blocks can be used to group and arrange the spectra of the M blocks in the current frame. By grouping and arranging the spectra of the M blocks, the arrangement order of the spectra of the M blocks in the current frame can be adjusted. After obtaining the spectrum to be encoded in the current frame, the spectrum to be encoded is encoded using an encoding neural network to obtain the spectrum encoding result, which can be carried in the bitstream. Therefore, in this embodiment of the application, the spectra of M blocks can be grouped and arranged according to the M transient identifiers in the current frame of the audio signal, thereby enabling grouping, arranging, and encoding of blocks with different transient identifiers, improving the encoding quality of the audio signal.

[0045] In another embodiment of this application, grouping information of M blocks of the current frame of the audio signal is obtained from the bitstream. This grouping information indicates M transient identifiers for the M blocks. The bitstream is decoded using a decoding neural network to obtain the decoded spectrum of the M blocks. The decoded spectrum of the M blocks is then reverse-grouped based on the grouping information to obtain the spectrum of the reverse-grouped M blocks. The reconstructed audio signal of the current frame is then obtained based on the spectrum of the reverse-grouped M blocks. Since the spectral encoding result included in the bitstream is grouped, decoding the bitstream yields the decoded spectrum of the M blocks. Reverse grouping then yields the spectrum of the reverse-grouped M blocks, thus obtaining the reconstructed audio signal of the current frame. During signal reconstruction, reverse grouping and decoding can be performed on blocks with different transient identifiers in the audio signal, thereby improving the audio signal reconstruction effect. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the composition structure of the audio processing system provided in the embodiments of this application;

[0047] Figure 2aA schematic diagram illustrating the application of the audio encoder and audio decoder provided in this application to a terminal device;

[0048] Figure 2b A schematic diagram illustrating the application of the audio encoder provided in this application to a wireless device or a core network device;

[0049] Figure 2c A schematic diagram illustrating the application of the audio decoder provided in this application embodiment to a wireless device or core network device;

[0050] Figure 3 A schematic diagram illustrating an audio signal encoding method provided in an embodiment of this application;

[0051] Figure 4 A schematic diagram illustrating an audio signal decoding method provided in an embodiment of this application;

[0052] Figure 5 A schematic diagram of an audio signal encoding and decoding system provided in an embodiment of this application;

[0053] Figure 6 A schematic diagram illustrating an audio signal encoding method provided in an embodiment of this application;

[0054] Figure 7 A schematic diagram illustrating an audio signal decoding method provided in an embodiment of this application;

[0055] Figure 8 A schematic diagram illustrating an audio signal encoding method provided in an embodiment of this application;

[0056] Figure 9 A schematic diagram illustrating an audio signal decoding method provided in an embodiment of this application;

[0057] Figure 10 This is a schematic diagram of the composition structure of an audio encoding device provided in an embodiment of this application;

[0058] Figure 11 This is a schematic diagram of the composition structure of an audio decoding device provided in an embodiment of this application;

[0059] Figure 12 This is a schematic diagram of the composition structure of another audio encoding device provided in the embodiments of this application;

[0060] Figure 13 This is a schematic diagram of the composition structure of another audio decoding device provided in an embodiment of this application. Detailed Implementation

[0061] The embodiments of this application will now be described with reference to the accompanying drawings.

[0062] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0063] Sound is a continuous wave produced by the vibration of an object. The object that produces vibrations and emits sound waves is called the sound source. As sound waves propagate through a medium (such as air, solids, or liquids), the auditory organs of humans or animals can perceive the sound.

[0064] Sound waves are characterized by pitch, intensity, and timbre. Pitch indicates the highness or lowness of a sound. Intensity indicates the loudness or volume of a sound. The unit of intensity is the decibel (dB). Timbre is also known as tone color.

[0065] The frequency of a sound wave determines its pitch. The higher the frequency, the higher the pitch. The number of times an object vibrates per second is called its frequency, and the unit of frequency is hertz (Hz). The human ear can distinguish sounds with frequencies between 20 Hz and 20,000 Hz.

[0066] The amplitude of a sound wave determines its intensity. The greater the amplitude, the greater the intensity. The closer to the sound source, the greater the intensity.

[0067] The waveform of a sound wave determines its timbre. Sound wave waveforms include square waves, sawtooth waves, sine waves, and pulse waves, among others.

[0068] Based on the characteristics of sound waves, sound can be divided into regular sound and irregular sound. Irregular sound refers to sound emitted by the irregular vibration of a sound source. Irregular sound is, for example, noise that affects people's work, study, and rest. Regular sound refers to sound emitted by the regular vibration of a sound source. Regular sound includes speech and musical tones. When sound is represented electronically, regular sound is an analog signal that varies continuously in the time and frequency domain. This analog signal can be called an audio signal. An audio signal is an information carrier that carries speech, music, and sound effects.

[0069] Because human hearing has the ability to distinguish the location of sound sources in space, when a listener hears a sound in space, in addition to being able to perceive the pitch, intensity, and timbre of the sound, they can also perceive the location of the sound.

[0070] Sound can also be categorized as mono or stereo. Mono has one sound channel, using a single microphone to pick up the sound and a single speaker to reproduce it. Stereo has multiple sound channels, with each channel transmitting a different sound waveform.

[0071] When the audio signal is a transient signal, the current encoding end does not extract the transient features and transmit them in the bitstream. These transient features are used to represent the changes in the spectrum of adjacent blocks in the transient frame of the audio signal. Therefore, when the signal is reconstructed at the decoding end, the transient features of the reconstructed audio signal cannot be obtained from the bitstream, resulting in poor audio signal reconstruction.

[0072] This application provides an audio processing technology, and more particularly an audio encoding technology for audio signals, to improve traditional audio encoding systems. Audio processing includes two parts: audio encoding and audio decoding. Audio encoding is performed on the source side, including encoding (e.g., compressing) the raw audio to reduce the amount of data required to represent the audio, thereby enabling more efficient storage and / or transmission. Audio decoding is performed on the destination side, including performing inverse processing relative to the encoder to reconstruct the original audio. The encoding and decoding parts are collectively referred to as encoding. The implementation of embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0073] The technical solutions of this application embodiment can be applied to various audio processing systems, such as... Figure 1 The diagram shown is a schematic representation of the composition of an audio processing system provided in this embodiment. The audio processing system 100 may include an audio encoding device 101 and an audio decoding device 102. The audio encoding device 101, also known as an audio signal encoding device, is used to generate a bitstream. This encoded bitstream is then transmitted to the audio decoding device 102 via an audio transmission channel. The audio decoding device 102, also known as an audio signal decoding device, receives the bitstream and performs its audio decoding function to obtain the reconstructed signal.

[0074] In the embodiments of this application, the audio encoding device can be applied to various terminal devices requiring audio communication, wireless devices requiring transcoding, and core network devices. For example, the audio encoding device can be an audio encoder for the aforementioned terminal devices, wireless devices, or core network devices. Similarly, the audio decoding device can be applied to various terminal devices requiring audio communication, wireless devices requiring transcoding, and core network devices. For example, the audio decoding device can be an audio decoder for the aforementioned terminal devices, wireless devices, or core network devices. For example, the audio encoder can include a wireless access network, a core network media gateway, a transcoding device, a media resource server, a mobile terminal, a fixed-line terminal, etc. The audio encoder can also be an audio encoder used in virtual reality (VR) streaming services.

[0075] In the application embodiment, taking the audio encoding module (audio encoding and audio decoding) applicable to virtual reality streaming (VR streaming) services as an example, the end-to-end audio signal encoding and decoding process includes: after the audio signal A passes through the acquisition module, it undergoes a preprocessing operation (audioPReprocessing). The preprocessing operation includes filtering out the low-frequency part in the signal, which can be based on 20Hz or 50Hz as the dividing point, extracting the location information in the signal, and then performing encoding processing (audio encoding), packaging (file / segment encapsulation), and then sending (delivery) to the decoding end. The decoding end first performs unpacking (file / segment decapsulation), then decoding (audio decoding), and performs binaural rendering processing on the decoded signal. The rendered signal is mapped onto the listener's headphones, which can be independent headphones or headphones on a glasses device.

[0076] like Figure 2aThe diagram illustrates the application of the audio encoder and audio decoder provided in this embodiment of the application to a terminal device. Each terminal device may include: an audio encoder, a channel encoder, an audio decoder, and a channel decoder. Specifically, the channel encoder is used for channel encoding of the audio signal, and the channel decoder is used for channel decoding of the audio signal. For example, the first terminal device 20 may include: a first audio encoder 201, a first channel encoder 202, a first audio decoder 203, and a first channel decoder 204. The second terminal device 21 may include: a second audio decoder 211, a second channel decoder 212, a second audio encoder 213, and a second channel encoder 214. The first terminal device 20 is connected to a wireless or wired first network communication device 22, and the first network communication device 22 and a wireless or wired second network communication device 23 are connected via a digital channel. The second terminal device 21 is connected to the wireless or wired second network communication device 23. The aforementioned wireless or wired network communication device can broadly refer to signal transmission devices, such as communication base stations, data exchange devices, etc.

[0077] In audio communication, the transmitting terminal device first acquires audio, encodes the acquired audio signal, and then performs channel coding before transmitting it over a digital channel via a wireless network or core network. The receiving terminal device, acting as the receiver, decodes the received signal to obtain the bitstream, then recovers the audio signal through audio decoding for playback.

[0078] like Figure 2b The diagram illustrates the application of the audio encoder provided in this embodiment of the application in a wireless device or core network device. The wireless device or core network device 25 includes: a channel decoder 251, other audio decoders 252, the audio encoder 253 provided in this embodiment of the application, and a channel encoder 254. The other audio decoders 252 refer to audio decoders other than the standard audio decoder. Within the wireless device or core network device 25, the signal entering the device is first channel-decoded using the channel decoder 251, then audio-decoded using the other audio decoder 252, then audio-encoded using the audio encoder 253 provided in this embodiment of the application, and finally channel-encoded using the channel encoder 254. After channel encoding, the signal is transmitted out. The other audio decoders 252 perform audio decoding on the bitstream decoded by the channel decoder 251.

[0079] like Figure 2cThe diagram illustrates the application of the audio decoder provided in this embodiment of the invention in a wireless device or core network device. The wireless device or core network device 25 includes: a channel decoder 251, an audio decoder 255 provided in this embodiment, other audio encoders 256, and a channel encoder 254. The other audio encoders 256 refer to audio encoders other than the standard audio encoder. Within the wireless device or core network device 25, the channel decoder 251 first performs channel decoding on the incoming signal. Then, the audio decoder 255 decodes the received audio encoded bitstream. Next, the other audio encoders 256 perform audio encoding. Finally, the channel encoder 254 performs channel encoding on the audio signal before transmission. If transcoding is required in the wireless device or core network device, corresponding audio encoding processing is necessary. The wireless device refers to radio frequency (RF) related equipment in communication, and the core network device refers to core network related equipment in communication.

[0080] In some embodiments of this application, the audio encoding device can be applied to various terminal devices requiring audio communication, wireless devices requiring transcoding, and core network devices. For example, the audio encoding device can be a multi-channel encoder of the aforementioned terminal device, wireless device, or core network device. Similarly, the audio decoding device can be applied to various terminal devices requiring audio communication, wireless devices requiring transcoding, and core network devices. For example, the audio decoding device can be a multi-channel decoder of the aforementioned terminal device, wireless device, or core network device.

[0081] This application first introduces an audio signal encoding method provided in its embodiments. This method can be executed by a terminal device, such as an audio signal encoding device (hereinafter referred to as an encoding end or encoder; for example, the encoding end can be an artificial intelligence (AI) encoder). Figure 3 The following describes the encoding process executed at the encoding end in this embodiment:

[0082] 301. Obtain M transient identifiers for M blocks based on the spectrum of M blocks in the current frame of the audio signal to be encoded; the M blocks include a first block, and the transient identifier of the first block is used to indicate that the first block is a transient block or to indicate that the first block is a non-transient block.

[0083] The encoding end first obtains the audio signal to be encoded, and then divides the audio signal into frames to obtain the current frame of the audio signal to be encoded. In the following embodiments, the encoding of the current frame is used as an example for illustration. The encoding of other frames of the audio signal to be encoded is similar to the encoding of the current frame.

[0084] After determining the current frame, the encoder performs windowing processing and time-frequency transformation on the current frame. If the current frame includes M blocks, the spectra of the M blocks of the current frame can be obtained, where M represents the number of blocks included in the current frame. In this embodiment, the value of M is not limited. For example, the encoder performs time-frequency transformation on the M blocks of the current frame to obtain the modified discrete cosine transform (MDCT) spectra of the M blocks. In subsequent embodiments, the spectra of the M blocks are used as the MDCT spectra, but it is not limited that the spectra of the M blocks can also be other spectra.

[0085] After obtaining the spectra of M blocks, the encoding end obtains M transient identifiers for each of the M blocks based on their spectra. The spectrum of each block is used to determine its transient identifier, and each block corresponds to one transient identifier. A block's transient identifier indicates the spectral changes of that block within the M blocks. For example, if a block among the M blocks is the first block, then that first block corresponds to one transient identifier.

[0086] In some embodiments of this application, the transient flag can be implemented in various ways. For example, the transient flag can indicate that the first block is a transient block, or it can indicate that the first block is a non-transient block. Specifically, a transient flag indicating a block is transient means that the spectrum of that block changes significantly compared to the spectra of other blocks in the M blocks; a non-transient flag indicating a block's spectrum does not change significantly compared to the spectra of other blocks in the M blocks. For example, the transient flag occupies 1 bit. If the transient flag value is 0, it is transient; if the transient flag value is 1, it is non-transient. Alternatively, if the transient flag value is 1, it is transient; if the transient flag value is 0, it is non-transient. This is not limited here.

[0087] 302. Obtain the grouping information of M blocks based on the M transient identifiers of M blocks.

[0088] After obtaining the M transient identifiers of M blocks, the encoding end uses the M transient identifiers of the M blocks to group the M blocks. Based on the M transient identifiers of the M blocks, the grouping information of the M blocks is obtained. The grouping information of the M blocks can represent the grouping method of the M blocks. The M transient identifiers of the M blocks are the basis for grouping the M blocks. For example, blocks with the same transient identifier can be grouped into one group, and blocks with different transient identifiers are grouped into different groups.

[0089] In some embodiments of this application, the grouping information of the M blocks can be implemented in various ways. The grouping information of the M blocks includes: the number of groups of the M blocks or a group number identifier, where the group number identifier indicates the number of groups. When the number of groups is greater than 1, the grouping information of the M blocks further includes: M transient identifiers of the M blocks; or, the grouping information of the M blocks includes: M transient identifiers of the M blocks. The grouping information of the M blocks can indicate the grouping status of the M blocks, so that the encoder can use this grouping information to group and arrange the spectrum of the M blocks.

[0090] For example, the grouping information for M blocks includes: the number of groups for the M blocks and the transient identifiers for the M blocks. These transient identifiers can also be called grouping flag information. Therefore, in this embodiment, the grouping information can include the number of groups and the grouping flag information. For example, the number of groups can be 1 or 2. The grouping flag information is used to indicate the transient identifiers of the M blocks.

[0091] For example, the grouping information for M blocks includes: transient identifiers for M blocks, which can also be called grouping flag information. Therefore, in this embodiment, the grouping information may include grouping flag information. For example, the grouping flag information is used to indicate the transient identifiers of the M blocks.

[0092] For example, the grouping information of M blocks includes: the number of groups of M blocks is 1. That is, when the number of groups is equal to 1, the grouping information of M blocks does not include M transient identifiers. However, when the number of groups is greater than 1, the grouping information of M blocks also includes M transient identifiers of M blocks.

[0093] For example, the number of groups in the grouping information of M blocks can be replaced with a group number identifier to indicate the number of groups. For example, a group number identifier of 0 indicates that the number of groups is 1, and a group number identifier of 1 indicates that the number of groups is 2.

[0094] In some embodiments of this application, the method executed at the encoding end further includes:

[0095] A1. Encode the grouping information of M blocks to obtain the grouping information encoding result;

[0096] A2. Write the encoded results of the grouping information into the bitstream.

[0097] After obtaining the grouping information of M blocks, the encoding end can carry this grouping information in the bitstream. First, the grouping information is encoded; the encoding method used is not limited here. By encoding the grouping information, the encoding result can be obtained, and this encoding result can be written into the bitstream, thus enabling the bitstream to carry the encoding result of the grouping information.

[0098] It should be noted that there is no specific order between step A2 and subsequent step 305. You can execute step 305 first and then step A2, or you can execute step A2 first and then step 305, or you can execute step A2 and step 305 simultaneously. There is no limitation here.

[0099] 303. Based on the grouping information of the M blocks, the spectra of the M blocks are grouped and arranged to obtain the spectrum to be encoded in the current frame.

[0100] The spectrum to be encoded can also be referred to as the spectrum of the M blocks after grouping and arranging.

[0101] After obtaining the grouping information of M blocks, the encoding end can use this information to group and arrange the spectra of the M blocks in the current frame. By grouping and arranging the spectra of the M blocks, the order of their arrangement in the current frame can be adjusted. This grouping and arrangement is based on the grouping information of the M blocks, which is obtained from the M transient identifiers of the M blocks. After grouping and arranging the M blocks, the spectra of the grouped M blocks are obtained. This grouped and arranged spectra are based on the M transient identifiers of the M blocks as the basis for grouping and sorting. By grouping and sorting, the encoding order of the spectra of the M blocks can be changed.

[0102] In some embodiments of this application, step 303 involves grouping and arranging the spectra of the M blocks according to their grouping information to obtain the spectrum to be encoded, including:

[0103] B1. Assign the spectrum of M blocks that is indicated as a transient block by M transient identifiers to the transient group, and assign the spectrum of M blocks that is indicated as a non-transient block by M transient identifiers to the non-transient group;

[0104] B2. Arrange the spectrum of the blocks in the transient group before the spectrum of the blocks in the non-transient group to obtain the spectrum to be encoded.

[0105] After obtaining the grouping information of M blocks, the encoder groups the M blocks based on their transient identifiers, thus obtaining transient groups and non-transient groups. Next, the positions of the M blocks in the spectrum of the current frame are arranged, placing the spectra of blocks in the transient groups before the spectra of blocks in the non-transient groups to obtain the spectrum to be encoded. That is, in the spectrum to be encoded, the spectra of all transient blocks are placed before the spectra of non-transient blocks, thereby adjusting the spectra of transient blocks to positions with higher coding importance, allowing the audio signal reconstructed after neural network encoding and decoding to better preserve transient features.

[0106] In some embodiments of this application, step 303 involves grouping and arranging the spectra of the M blocks according to their grouping information to obtain the spectrum to be encoded for the current frame, including:

[0107] C1. Arrange the spectra of the M blocks that are marked as transient blocks by the M transient flags before the spectra of the M blocks that are marked as non-transient blocks by the M transient flags to obtain the spectrum to be encoded in the current frame.

[0108] In this process, after obtaining the grouping information of M blocks, the encoder determines the transient identifier of each block within the M blocks based on this information. First, P transient blocks and Q non-transient blocks are identified from the M blocks, then M = P + Q. The spectra of the blocks identified as transient by the M transient identifiers are arranged before the spectra of the blocks identified as non-transient by the M transient identifiers to obtain the spectrum to be encoded for the current frame. That is, the spectra of all transient blocks in the spectrum to be encoded are placed before the spectra of the non-transient blocks, thus allowing the spectra of transient blocks to be adjusted to positions of higher coding importance, enabling the audio signal reconstructed after neural network encoding and decoding to better preserve transient characteristics.

[0109] 304. Use a coding neural network to encode the spectrum to be encoded to obtain the spectrum encoding result.

[0110] 305. Write the spectrum encoding result into the bitstream.

[0111] In this embodiment of the application, after the encoding end obtains the spectrum to be encoded in the current frame, it can use an encoding neural network to encode it to generate a spectrum encoding result, and then write the spectrum encoding result into the bit stream. The encoding end can then send the bit stream to the decoding end.

[0112] One possible approach is to use the spectrum to be encoded as input data to the encoding neural network at the encoding end. Alternatively, the spectrum to be encoded can undergo other processing before being used as input data for the encoding neural network. After processing by the encoding neural network, latent variables can be generated, representing the characteristics of the spectra of the M blocks after grouping and arranging.

[0113] In some embodiments of this application, before step 304 encodes the spectrum to be encoded using the coding neural network, the method performed at the encoding end further includes:

[0114] D1. Perform intra-group interleaving on the spectrum to be encoded to obtain the spectrum of M blocks with intra-group interleaving.

[0115] In this implementation scenario, step 304 uses an encoding neural network to encode the spectrum to be encoded, including:

[0116] E1. Encode the spectra of M blocks that have undergone intra-group interleaving using an encoding neural network.

[0117] In this process, after obtaining the spectrum to be encoded in the current frame, the encoder can first perform intra-group interleaving based on the grouping of M blocks, thereby obtaining the spectrum of the M interleaved blocks. The spectrum of the M interleaved blocks can then be used as the input data for the encoding neural network. Intra-group interleaving can also reduce the side information in the encoding, improving coding efficiency.

[0118] In some embodiments of this application, the number of blocks identified as transient by M transient flags is P, and the number of blocks identified as non-transient by M transient flags is Q, where M = P + Q. The values ​​of P and Q are not limited in these embodiments.

[0119] Specifically, step D1 involves intra-group interleaving of the spectrum to be encoded, including:

[0120] D11. Interleave the spectra of P blocks to obtain the interleaved spectra of P blocks;

[0121] D12. Interleave the spectra of Q blocks to obtain the interleaved spectra of Q blocks.

[0122] Interleaving the spectra of P blocks includes treating the spectra of the P blocks as a whole and interleaving them; similarly, interleaving the spectra of Q blocks includes treating the spectra of the Q blocks as a whole and interleaving them.

[0123] In the case of executing steps D11 and D12, step E1 encodes the spectra of the M blocks of intra-group interleaving using an encoding neural network, including:

[0124] The spectrum of P blocks of interleaved processing and the spectrum of Q blocks of interleaved processing are encoded using an encoding neural network.

[0125] In steps D11 to D12, the encoder can perform interleaving processing separately for transient and non-transient groups, thereby obtaining the spectra of P interleaved blocks and Q interleaved blocks. The spectra of the P interleaved blocks and the spectra of the Q interleaved blocks can be used as input data for the coding neural network. Intra-group interleaving can also reduce side information in the encoding, improving coding efficiency.

[0126] In some embodiments of this application, before step 301 obtains the M transient identifiers of the M blocks based on the spectra of the M blocks of the current frame of the audio signal to be encoded, the method performed by the encoding end further includes:

[0127] F1. Get the window type of the current frame, which is either a short window or a non-short window.

[0128] F2. When the window type is short window type, the step of obtaining M transient identifiers of M blocks based on the spectrum of M blocks of the current frame of the audio signal to be encoded is executed.

[0129] Before executing step 301 at the encoding end, the encoding end can first determine the window type of the current frame. This window type can be a short window type or a non-short window type. For example, the encoding end determines the window type based on the current frame of the audio signal to be encoded. A short window can also be called a short frame, and a non-short window can also be called a non-short frame. When the window type is a short window type, the aforementioned step 301 is triggered. In this embodiment, the aforementioned encoding scheme can only be executed when the window type of the current frame is a short window type, thus achieving encoding when the audio signal is a transient signal.

[0130] In some embodiments of this application, when the encoding end performs the aforementioned steps F1 and F2, the method performed by the encoding end further includes:

[0131] G1. Encode the window type to obtain the window type encoding result;

[0132] G2. Write the window type encoding result to the bitstream.

[0133] After obtaining the window type of the current frame, the encoding end can carry the window type in the bitstream. First, the window type is encoded; the encoding method used for the window type is not limited here. By encoding the window type, the window type encoding result can be obtained, and this result can be written into the bitstream, thus enabling the bitstream to carry the window type encoding result.

[0134] In some embodiments of this application, step 301, obtaining M transient identifiers for the M blocks based on the spectra of the M blocks of the current frame of the audio signal to be encoded, includes:

[0135] H1. Obtain the M spectral energies of the M blocks based on the spectra of the M blocks;

[0136] H2. Obtain the average spectral energy of M blocks based on the M spectral energies;

[0137] H3. Obtain M transient identifiers for M blocks based on the M spectral energies and the average spectral energy.

[0138] After obtaining M spectral energies, the encoding end can average these M spectral energies to obtain an average spectral energy value, or remove the maximum value or several of the largest values ​​from the M spectral energies before averaging to obtain the average spectral energy value. By comparing the spectral energy of each block in the M spectral energies with the average spectral energy value, the variation of the spectrum of each block compared to the spectra of other blocks in the M blocks is determined, thereby obtaining M transient identifiers for the M blocks. A transient identifier for a block can be used to represent the transient characteristics of that block. In this embodiment, the transient identifier of each block can be determined by comparing its spectral energy with the average spectral energy value, enabling the grouping information of a block to be determined by its transient identifier.

[0139] Furthermore, in some embodiments of this application, when the spectral energy of the first block is greater than K times the average spectral energy, the transient identifier of the first block indicates that the first block is a transient block; or,

[0140] When the spectral energy of the first block is less than or equal to K times the average spectral energy, the transient flag of the first block indicates that the first block is a non-transient block;

[0141] Where K is a real number greater than or equal to 1.

[0142] The value of K can take many forms, which are not limited here. Taking the determination of the transient identifier of the first block out of M blocks as an example, when the spectral energy of the first block is greater than K times the average spectral energy, it indicates that the spectral change of the first block is too large compared to the other M blocks, and the transient identifier of the first block indicates that the first block is a transient block. When the spectral energy of the first block is less than or equal to K times the average spectral energy, it indicates that the spectral change of the first block is not significant compared to the other M blocks, and the transient identifier of the first block indicates that the first block is a non-transient block.

[0143] Without limitation, the encoder can also obtain the M transient identifiers of the M blocks in other ways, such as obtaining the difference or ratio between the spectral energy of the first block and the average spectral energy, and determine the M transient identifiers of the M blocks based on the obtained difference or ratio.

[0144] As illustrated by the foregoing embodiments, the encoding end obtains M transient identifiers for M blocks based on the spectra of M blocks in the current frame of the audio signal to be encoded. After obtaining the grouping information of the M blocks based on the M transient identifiers, the grouping information of the M blocks can be used to group and arrange the spectra of the M blocks in the current frame. By grouping and arranging the spectra of the M blocks, the arrangement order of the spectra of the M blocks in the current frame can be adjusted. After obtaining the spectrum to be encoded, the spectrum to be encoded is encoded using an encoding neural network to obtain the spectrum encoding result. The spectrum encoding result can be carried through the bitstream. Therefore, in this embodiment, the spectra of the M blocks can be grouped and arranged according to the M transient identifiers in the current frame of the audio signal, thereby enabling the grouping and encoding of blocks with different transient identifiers and improving the encoding quality of the audio signal.

[0145] This application also provides an audio signal decoding method, which can be executed by a terminal device, such as an audio signal decoding device (hereinafter referred to as a decoding end or decoder, for example, the decoding end can be an AI decoder). Figure 4 As shown, the method executed at the decoding end in this embodiment mainly includes:

[0146] 401. Obtain the grouping information of the M blocks of the current frame of the audio signal from the bitstream. The grouping information is used to indicate the M transient identifiers of the M blocks.

[0147] The decoding end receives the bitstream sent by the encoding end. The encoding end writes the packet information encoding result into the bitstream. The decoding end parses the bitstream to obtain the packet information of M blocks of the current frame of the audio signal. Based on the packet information of these M blocks, the decoding end can determine the M transient identifiers of the M blocks. For example, the packet information may include: the number of blocks and the block flag information. Alternatively, the packet information may include the block flag information, as detailed in the aforementioned embodiment of the encoding end.

[0148] 402. Use a decoding neural network to decode the bitstream to obtain the decoded spectrum of M blocks.

[0149] After obtaining the bitstream, the decoding end uses a decoding neural network to decode the bitstream and obtain the decoded spectrum of M blocks. Since the encoding end encodes the spectrum of the M blocks after grouping and arranging them, the encoding end carries the spectrum encoding result in the bitstream. The decoded spectrum of the M blocks corresponds to the spectrum of the M blocks after grouping and arranging by the encoding end. The execution process of the decoding neural network is the reverse of that of the encoding neural network at the encoding end. Through decoding, the reconstructed spectrum of the M blocks after grouping and arranging can be obtained.

[0150] 403. Perform reverse grouping and rearrangement processing on the decoded spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum of the reverse grouping and rearrangement processing of the M blocks.

[0151] The decoding end obtains the grouping information of M blocks. The decoding end also obtains the decoded spectrum of M blocks through the bitstream. Since the encoding end has performed grouping and arrangement processing on the spectrum of M blocks, the decoding end needs to perform the reverse process of the encoding end. Therefore, the decoding spectrum of M blocks is reversed according to the grouping information of M blocks to obtain the spectrum of M blocks after reverse grouping and arrangement processing. This reverse grouping and arrangement processing is the reverse of the grouping and arrangement processing of the encoding end.

[0152] 404. Obtain the reconstructed audio signal of the current frame based on the spectrum of the M blocks after inverse grouping and arrangement processing.

[0153] After obtaining the spectrum of the M blocks after inverse grouping and arrangement processing, the encoding end can transform the spectrum of the M blocks after inverse grouping and arrangement processing from the frequency domain to the time domain to obtain the reconstructed audio signal of the current frame.

[0154] In some embodiments of this application, before step 403 performs the reverse grouping and rearrangement process on the decoded spectra of the M blocks according to the grouping information of the M blocks, the method executed at the decoding end further includes:

[0155] I1. Perform intra-group deinterleaving processing on the decoded spectra of M blocks to obtain the intra-group deinterleaving processed spectra of M blocks;

[0156] Step 403 involves reversing the grouping of the decoded spectra of the M blocks based on their grouping information, including:

[0157] J1. Based on the grouping information of M blocks, the spectra of the intra-group deinterleaving processing of M blocks are reverse-grouped and arranged.

[0158] The intra-group deinterleaving performed at the decoding end is the reverse process of intra-group interleaving at the encoding end, which will not be described in detail here.

[0159] In some embodiments of this application, the number of M blocks that are indicated as transient blocks by M transient flags is P, and the number of M blocks that are indicated as non-transient blocks by M transient flags is Q, where M = P + Q;

[0160] Step I1 performs intra-group deinterleaving processing on the decoded spectra of the M blocks, including:

[0161] I11. Deinterleave the decoded spectra of the P blocks; and,

[0162] I12. Deinterleave the decoding spectra of the Q blocks.

[0163] Specifically, deinterleaving the spectra of P blocks includes treating the spectra of the P blocks as a whole for deinterleaving; similarly, deinterleaving the spectra of Q blocks includes treating the spectra of the Q blocks as a whole for deinterleaving.

[0164] The encoding end can perform interleaving processing separately for transient and non-transient groups, thereby obtaining the spectra of P interleaved blocks and Q interleaved blocks. The spectra of the P interleaved blocks and the Q interleaved blocks can be used as input data for the coding neural network. Intra-group interleaving can also reduce side information in the encoding, improving coding efficiency. Since the encoding end performs intra-group interleaving, the decoding end needs to perform the corresponding reverse process, i.e., the decoding end can perform deinterleaving.

[0165] In some embodiments of this application, the number of M blocks after reconstruction and grouping that are indicated as transient blocks by M transient identifiers is P, and the number of M blocks that are indicated as non-transient blocks by M transient identifiers is Q, where M = P + Q;

[0166] Step 403 involves reversing the grouping of the decoded spectra of the M blocks based on their grouping information, including:

[0167] K1. Obtain the indices of P blocks based on the grouping information of M blocks;

[0168] K2. Obtain the indices of Q blocks based on the grouping information of M blocks;

[0169] K3. Perform reverse grouping and rearrangement of the decoded spectra of the M blocks according to the indices of the P blocks and the indices of the Q blocks.

[0170] Before the encoder groups and arranges the spectra of the M blocks, the indices of the M blocks are continuous, for example, from 0 to M-1. After the encoder groups and arranges the blocks, the indices of the M blocks are no longer continuous. Based on the grouping information of the M blocks, the decoder can obtain the indices of P blocks and Q blocks in the reconstructed grouped and arranged M blocks. Through reverse grouping and arrangement, the indices of the M blocks can be restored to be continuous.

[0171] In some embodiments of this application, the method executed at the decoding end further includes:

[0172] L1. Obtain the window type of the current frame from the bitstream, which is either a short window or a non-short window;

[0173] L2. The step of obtaining the grouping information of the M blocks of the current frame from the bitstream is only executed when the window type of the current frame is short window type.

[0174] In this embodiment, the aforementioned encoding scheme can only be executed when the window type of the current frame is a short window type, thus achieving encoding when the audio signal is a transient signal. The decoding end performs the reverse process of the encoding end; therefore, the decoding end can also first determine the window type of the current frame, which can be a short window type or a non-short window type. For example, the decoding end obtains the window type of the current frame from the bitstream. A short window can also be called a short frame, and a non-short window can also be called a non-short frame. When the window type is a short window type, the aforementioned step 401 is triggered.

[0175] In some embodiments of this application, the grouping information of M blocks includes: the number of groups of M blocks or the group number identifier, wherein the group number identifier is used to indicate the number of groups. When the number of groups is greater than 1, the grouping information of M blocks also includes: M transient identifiers of M blocks.

[0176] Alternatively, the grouping information for the M blocks includes: M transient identifiers for the M blocks.

[0177] As illustrated by the foregoing examples of the decoding end, the grouping information of M blocks of the current frame of the audio signal is obtained from the bitstream. This grouping information indicates the M transient identifiers of the M blocks. The bitstream is decoded using a decoding neural network to obtain the decoded spectrum of the M blocks. Based on the grouping information of the M blocks, the decoded spectrum of the M blocks is reverse-grouped to obtain the spectrum of the reverse-grouped M blocks. The reconstructed audio signal of the current frame is then obtained based on the spectrum of the reverse-grouped M blocks. Since the spectral encoding result included in the bitstream is grouped, decoding the bitstream yields the decoded spectrum of the M blocks. Then, through reverse grouping, the spectrum of the reverse-grouped M blocks can be obtained, thus obtaining the reconstructed audio signal of the current frame. During signal reconstruction, reverse grouping and decoding can be performed on blocks with different transient identifiers in the audio signal, thereby improving the audio signal reconstruction effect.

[0178] To facilitate a better understanding and implementation of the above-described solutions in the embodiments of this application, specific examples of corresponding application scenarios are provided below.

[0179] like Figure 5 The diagram shown is a schematic of the system architecture provided in this application embodiment for application in the field of broadcasting and television. This application embodiment can also be applied to live broadcasting and post-production scenarios of broadcasting and television, or to a three-dimensional sound codec in terminal media playback.

[0180] In live streaming scenarios, the 3D audio signal generated from the live program's 3D audio is encoded using the 3D audio encoding method described in this application to obtain a bitstream. This bitstream is then transmitted to the user side via a broadcast network. The 3D audio signal is decoded and reconstructed by a 3D audio decoder in a set-top box, and then played back by a speaker system. In post-production scenarios, the 3D audio signal generated from the post-production program's 3D audio is encoded using the 3D audio encoding method described in this application to obtain a bitstream. This bitstream is transmitted to the user side via a broadcast network or the Internet. The 3D audio signal is decoded and reconstructed by a 3D audio decoder in a network receiver or mobile terminal, and then played back by a speaker system or headphones.

[0181] This application provides an audio codec, which may specifically include a wireless access network, a media gateway in the core network, a transcoding device, a media resource server, a mobile terminal, a fixed-line terminal, etc. It can also be applied to audio codecs in broadcast television or terminal media playback, and VR streaming services.

[0182] The application scenarios of the encoding and decoding ends in the embodiments of this application will be described below.

[0183] like Figure 6 As shown, the encoder proposed in this application embodiment performs the following audio signal encoding method, including:

[0184] S11. Determine the window type for the current frame.

[0185] Obtain the audio signal of the current frame, determine the window type of the current frame based on the audio signal, and write the window type into the bitstream.

[0186] One specific implementation method includes the following three steps:

[0187] 1) Perform frame-by-frame processing on the audio signal to be encoded to obtain the audio signal of the current frame.

[0188] For example, if the frame length of the current frame is L samples, then the audio signal of the current frame is an L-point time-domain signal.

[0189] 2) Perform transient detection based on the audio signal of the current frame to determine the transient information of the current frame.

[0190] There are various methods for transient detection, and this application does not limit them. The transient information of the current frame may include one or more of the following: an identifier indicating whether the current frame is a transient signal, the location where the transient occurs in the current frame, and parameters characterizing the degree of transientity. The degree of transientity may be the level of transient energy, or the ratio of the signal energy at the location where the transient occurs to the signal energy at adjacent non-transient locations.

[0191] 3) Based on the transient information of the current frame, determine the window type of the current frame, encode the window type of the current frame, and write the encoding result into the bitstream.

[0192] If the transient information of the current frame indicates that the current frame is a transient signal, then the window type of the current frame is a short window.

[0193] If the transient information of the current frame indicates that the current frame is a non-transient signal, then the window type of the current frame is another window type excluding short windows. This application does not limit other window types; for example, other window types may include: long windows, cut-in windows, cut-out windows, etc.

[0194] S12. If the window type of the current frame is short window, perform short windowing processing on the audio signal of the current frame and perform time-frequency transformation to obtain the MDCT spectrum of the M blocks of the current frame.

[0195] If the window type of the current frame is short window, perform short windowing processing on the audio signal of the current frame and perform time-frequency transformation to obtain the MDCT spectrum of M blocks.

[0196] For example, if the current frame's window type is short window, M overlapping short window functions are used for windowing to obtain M blocks of windowed audio signals, where M is a positive integer greater than or equal to 2. For instance, the window length of the short window function is 2L / M, where L is the frame length of the current frame, and the overlap length is L / M. For example, if M equals 8 and L equals 1024, the window length of the short window function is 256 samples, and the overlap length is 128 samples.

[0197] The M windowed audio signals are subjected to time-frequency transformation to obtain the MDCT spectrum of the M blocks in the current frame.

[0198] For example, the length of the windowed audio signal of the current block is 256 samples. After MDCT transformation, 128 MDCT coefficients are obtained, which is the MDCT spectrum of the current block.

[0199] S13. Based on the MDCT spectrum of M blocks, obtain the number of blocks and block flag information of the current frame, encode the number of blocks and block flag information of the current frame, and write the encoding result into the bitstream.

[0200] Before obtaining the number of packets and packet flag information of the current frame in step S13, in one implementation: First, the MDCT spectra of M blocks are interleaved to obtain the interleaved MDCT spectra of M blocks; next, the MDCT spectra of the interleaved M blocks are encoded and preprocessed to obtain the preprocessed MDCT spectra; then, the preprocessed MDCT spectra are deinterleaved to obtain the deinterleaved MDCT spectra of M blocks; finally, the number of packets and packet flag information of the current frame are determined based on the deinterleaved MDCT spectra of the M blocks.

[0201] Interleaving the MDCT spectra of M blocks involves interleaving the M MDCT spectra of length L / M into a single MDCT spectrum of length L. The M spectral coefficients at frequency i in the M blocks' MDCT spectra are arranged sequentially from 0 to M-1 according to their block numbers. Then, the M spectral coefficients at frequency i+1 in the M blocks' MDCT spectra are arranged sequentially from 0 to M-1 according to their block numbers, where i ranges from 0 to L / M-1.

[0202] The encoding preprocessing operations may include frequency domain noise shaping (FDNS), temporal noise shaping (TNS), and bandwidth extension (BWE), etc., which are not limited here.

[0203] Deinterleaving is the reverse process of interleaving. The preprocessed MDCT spectrum has a length of L. This preprocessed MDCT spectrum of length L is divided into M MDCT spectra of length L / M. The MDCT spectra in each block are arranged in ascending order of frequency, thus obtaining the MDCT spectra of the M blocks after deinterleaving. Preprocessing the interleaved spectrum reduces the encoding of side information, thereby reducing the bit occupation of side information and improving coding efficiency.

[0204] The number of packets and packet flag information of the current frame are determined based on the MDCT spectra of the M blocks after deinterleaving. The specific method includes the following three steps:

[0205] a) Calculate the MDCT spectral energy of M blocks.

[0206] Assuming the MDCT spectral coefficients of the M blocks after deinterleaving are mdctSpectrum[8]

[128] , calculate the MDCT spectral energy of each block, denoted as enerMdct[8]. Here, 8 is the value of M, and 128 represents the number of MDCT coefficients in a block.

[0207] b) Calculate the average MDCT spectral energy based on the MDCT spectral energy of the M blocks. This mainly includes the following two methods:

[0208] Method 1: Directly calculate the average value of the MDCT spectrum energy of M blocks, i.e., the average value of enerMdct[8], as the average value of the MDCT spectrum energy avgEner.

[0209] Method 2: Determine the block with the highest MDCT spectral energy among the M blocks; calculate the average MDCT spectral energy of the other M-1 blocks excluding the block with the highest energy, and use this average MDCT spectral energy as avgEner. Alternatively, calculate the average MDCT spectral energy of the other blocks excluding the blocks with the highest energy, and use this average MDCT spectral energy as avgEner.

[0210] c) Based on the MDCT spectral energy of the M blocks and the average MDCT spectral energy, determine the number of blocks and the block flag information of the current frame, and write them into the bitstream.

[0211] Specifically, this can be done by comparing the MDCT spectral energy of each block with the average MDCT spectral energy. If the MDCT spectral energy of the current block is greater than K times the average MDCT spectral energy, then the current block is a transient block, and its transient flag is 0; otherwise, the current block is a non-transient block, and its non-transient flag is 1. Here, K is greater than or equal to 1, for example, K = 2. Based on the transient flags of each block, the M blocks are grouped, and the number of groups and grouping information are determined. Blocks with the same transient flag value are grouped together, and the M blocks are divided into N groups, where N is the number of groups. The grouping information consists of the transient flag values ​​of each of the M blocks.

[0212] For example, transient blocks form transient groups, and non-transient blocks form non-transient groups. Specifically, if the transient identifiers of each block are not completely identical, the number of groups in the current frame, numGroups, is 2; otherwise, it is 1. The number of groups can be represented by a group number identifier. For example, a group number identifier of 1 indicates that the number of groups in the current frame is 2; a group number identifier of 0 indicates that the number of groups in the current frame is 1. The group indicator information groupIndicator of the current frame is determined based on the transient identifiers of M blocks. For example, the group indicator information groupIndicator of the current frame is formed by arranging the transient identifiers of the M blocks in sequence.

[0213] Before obtaining the number of packets and packet flag information in step S13, another implementation method is: without performing interleaving and deinterleaving processing on the MDCT spectrum of the M blocks, directly determine the number of packets and packet flag information of the current frame based on the MDCT spectrum of the M blocks, encode the number of packets and packet flag information of the current frame, and write the encoding result into the bitstream.

[0214] Determining the number of packets and packet flag information of the current frame based on the MDCT spectrum of M blocks is similar to determining the number of packets and packet flag information of the current frame based on the MDCT spectrum of the M blocks after deinterleaving, and will not be repeated here.

[0215] Write the number of packets and packet flag information of the current frame into the bitstream.

[0216] Furthermore, the non-transient group can be further divided into two or more other groups, which is not limited in the embodiments of this application. For example, the non-transient group can be divided into a harmonic group and a non-harmonic group.

[0217] S14. Arrange the MDCT spectra of the M blocks into groups according to the number of groups and group flag information of the current frame to obtain the grouped MDCT spectrum. The grouped MDCT spectrum is the spectrum to be encoded in the current frame.

[0218] If the current frame has 2 groups, then the audio signal spectra of the M blocks in the current frame need to be grouped and arranged. The arrangement is as follows: several blocks belonging to the transient group are moved to the front, and several blocks belonging to the non-transient group are moved to the back. The encoder's neural network performs better on the spectra placed earlier; therefore, moving the transient blocks to the front ensures their encoding quality, preserving more spectral details and improving overall encoding quality.

[0219] The MDCT spectra of the M blocks in the current frame can be grouped and arranged according to the number of groups and group flag information of the current frame. Alternatively, the MDCT spectra of the M blocks after deinterleaving of the current frame can be grouped and arranged according to the number of groups and group flag information of the current frame.

[0220] S15. Encode the grouped MDCT spectrum using an encoding neural network and write it into the code stream.

[0221] The grouped MDCT spectra are first interleaved within each group to obtain intra-group interleaved MDCT spectra. Then, an encoding neural network is used to encode the intra-group interleaved MDCT spectra. The intra-group interleaving process is similar to the interleaving process performed on the M blocks of MDCT spectra before obtaining the group number and group identifier information, except that the interleaved objects are MDCT spectra belonging to the same group. For example, MDCT spectrum blocks belonging to transient groups are interleaved. MDCT spectrum blocks belonging to non-transient groups are also interleaved.

[0222] The encoded neural network is pre-trained, and this application does not limit the specific network structure and training method of the encoded neural network. For example, the encoded neural network can be a fully connected network or a convolutional neural network (CNN).

[0223] like Figure 7 As shown, the decoding process corresponding to the encoding end includes:

[0224] S21. Decode the received bitstream to obtain the window type of the current frame.

[0225] S22. If the window type of the current frame is a short window, then the number of packets and packet flag information are obtained by decoding the received bitstream.

[0226] It can parse the packet count identifier information in the bitstream and determine the packet count of the current frame based on the packet count identifier information. For example, a packet count identifier of 1 indicates that the current frame has 2 packets; a packet count identifier of 0 indicates that the current frame has 1 packet.

[0227] If the number of packets in the current frame is greater than 1, the packet flag information can be obtained by decoding the received bitstream.

[0228] Decoding the received bitstream yields the block identifier information, which can be achieved by reading M bits of block identifier information from the bitstream. The value of the i-th bit of the block identifier information determines whether the i-th block is a transient block. If the value of the i-th bit is 0, the i-th block is a transient block; if the value of the i-th bit is 1, the i-th block is a non-transient block.

[0229] S23. Based on the received bitstream, use a decoding neural network to obtain the decoded MDCT spectrum.

[0230] The decoding process at the decoding end corresponds to the encoding process at the encoding end. Specific steps include:

[0231] First, based on the received bitstream decoding, the decoded MDCT spectrum is obtained using a decoding neural network.

[0232] Then, based on the number of groups and group identifier information, the decoded MDCT spectra belonging to the same group can be determined. Intra-group deinterleaving is then performed on the MDCT spectra belonging to the same group to obtain the intra-group deinterleaved MDCT spectra. This intra-group deinterleaving process is the same as the deinterleaving process performed on the MDCT spectra of the M interleaved blocks before the encoder obtains the number of groups and group identifier information.

[0233] S24. Based on the number of groups and the grouping information, perform reverse grouping and rearrangement processing on the MDCT spectrum after deinterleaving within the group to obtain the MDCT spectrum after reverse grouping and rearrangement.

[0234] If the number of packets in the current frame is greater than 1, then the MDCT spectrum processed by the deinterleaving within the group needs to be reversed according to the packet flag information. The reverse packet reordering process at the decoding end is the reverse process of the packet reordering process at the encoding end.

[0235] For example, suppose the MDCT spectrum of the intra-group deinterleaving process consists of M L / M point MDCT spectrum blocks. Based on the group flag information, the block index idx0(i) of the i-th transient block is determined. The MDCT spectrum of the i-th block in the intra-group deinterleaving MDCT spectrum is used as the MDCT spectrum of the idx0(i)-th block in the inverse grouping MDCT spectrum. The block index idx0(i) of the i-th transient block is the block index corresponding to the i-th block with a flag value of 0 in the group flag information, where i starts from 0. The number of transient blocks is the number of bits with a flag value of 0 in the group flag information, denoted as num0. After processing the transient blocks, the non-transient blocks need to be processed. Based on the group flag information, the block index idx1(j) of the j-th non-transient block is determined. The MDCT spectrum of the num0+j-th block in the intra-group deinterleaving MDCT spectrum is used as the MDCT spectrum of the idx1(j)-th block in the inverse grouping MDCT spectrum. The block index idx1(j) of the j-th non-transient block is the block index corresponding to the j-th block with a flag value of 1 in the group flag information, where j starts from 0.

[0236] S25. Obtain the reconstructed audio signal of the current frame based on the MDCT spectrum processed by inverse grouping.

[0237] Based on the MDCT spectrum obtained from the inverse grouping and arrangement processing, the reconstructed audio signal is obtained. One specific implementation is as follows: First, the MDCT spectra of the M blocks of the inverse grouping and arrangement processing are interleaved to obtain the interleaved MDCT spectra of the M blocks. Next, the interleaved MDCT spectra of the M blocks are subjected to decoding post-processing operations, such as inverse TNS, inverse FDNS, BWE processing, etc. The decoding post-processing corresponds one-to-one with the encoding pre-processing methods at the encoding end to obtain the decoded MDCT spectrum. Then, the decoded MDCT spectrum is de-interleaved to obtain the de-interleaved MDCT spectrum of the M blocks. Finally, the de-interleaved MDCT spectra of the M blocks are transformed from the frequency domain to the time domain, and after dewindowing and superposition addition processing, the reconstructed audio signal is obtained.

[0238] Another specific way to obtain the reconstructed audio signal based on the MDCT spectrum processed by inverse grouping is to transform the MDCT spectrum of each of the M blocks from the frequency domain to the time domain, and then perform dewindowing and overlay addition processing to obtain the reconstructed audio signal.

[0239] like Figure 8 As shown, the audio signal encoding methods executed by the encoding end include:

[0240] S31. Perform frame segmentation on the input signal to obtain the input signal of the current frame.

[0241] For example, if the frame length is 1024, the input signal for the current frame is a 1024-point audio signal.

[0242] S32. Perform transient detection based on the input signal of the current frame to obtain the transient detection result.

[0243] For example, the input signal of the current frame is divided into L blocks, and the signal energy in each block is calculated. If the signal energy in adjacent blocks changes abruptly, the current frame is considered a transient signal. For example, L is a positive integer greater than 2, and L = 8 can be chosen. If the difference in signal energy between adjacent blocks is greater than a preset threshold, the current frame is considered a non-transient signal.

[0244] S33. Determine the window type of the current frame based on the transient detection results.

[0245] If the transient detection result of the current frame is a transient signal, then the window type of the current frame is a short window; otherwise, it is a long window.

[0246] In addition to short windows and long windows, the window type of the current frame can also include in-window and out-window. Let the frame number of the current frame be i. Based on the transient detection results of frames i-1 and i-2 and the transient detection result of the current frame, the window type of the current frame is determined.

[0247] If the transient detection results of the i-th frame, the (i-1)-th frame, and the (i-2)-th frame are all non-transient signals, then the window type of the i-th frame is a long window.

[0248] If the transient detection result of frame i is a transient signal, and the transient detection results of frames i-1 and i-2 are non-transient signals, then the window type of frame i is an in-line window.

[0249] If the transient detection results of frame i and frame i-1 are non-transient signals, and the transient detection result of frame i-2 is a transient signal, then the window type of frame i is a cutout window.

[0250] If the transient detection results of the i-th frame, the (i-1)-th frame, and the (i-2)-th frame are any other than the three cases mentioned above, then the window type of the i-th frame is a short window.

[0251] S34. Based on the window type of the current frame, perform windowing and time-frequency transformation processing to obtain the MDCT spectrum of the current frame.

[0252] Based on the types of long windows, cut-in windows, cut-out windows, and short windows, windowing and MDCT transformation are performed respectively: For long windows, cut-in windows, and cut-out windows, if the signal length after windowing is 2048, then 1024 MDCT coefficients are obtained; for short windows, 8 overlapping short windows with a length of 256 are added, and each short window obtains 128 MDCT coefficients. The 128 MDCT coefficients of each short window are called a block, for a total of 1024 MDCT coefficients.

[0253] Determine whether the window type of the current frame is a short window. If yes, proceed to step S35; otherwise, proceed to step S312.

[0254] S35. If the window type of the current frame is a short window, perform interleaving processing on the MDCT spectrum of the current frame to obtain the interleaved MDCT spectrum.

[0255] If the window type of the current frame is a short window, the MDCT spectra of the 8 blocks are interleaved, that is, the 8 128-dimensional MDCT spectra are interleaved into an MDCT spectrum of length 1024.

[0256] The interleaved spectrum can take the form of: block 0bin 0, block 1bin 0, block 2bin 0, ..., block7bin 0, block 0bin 1, block 1, bin 1, block 2bin 1, ..., block 7bin 1, ...

[0257] Here, block 0bin 0 represents the 0th frequency point of the 0th block.

[0258] S36. Perform encoding preprocessing on the interleaved MDCT spectrum to obtain the preprocessed MDCT spectrum.

[0259] Preprocessing can include FDNS, TNS, BWE, and other processing.

[0260] S37. Deinterleave the preprocessed MDCT spectrum to obtain M blocks of MDCT spectrum.

[0261] Deinterleaving is performed in the reverse manner of step S35 to obtain MDCT spectra of 8 blocks, each block having 128 points.

[0262] S38. Determine the grouping information based on the MDCT spectra of M blocks.

[0263] The information may include the number of groups numGroups and the group indicator information groupIndicator. Based on the MDCT spectra of the M blocks, the specific scheme for determining the group information can be any of the aforementioned steps S13 performed by the encoder. For example, if the MDCT spectral coefficients of the 8 blocks in the short frame are mdctSpectrum[8]

[128] , then the MDCT spectral energy of each block is calculated and denoted as enerMdct[8]. The average value of the MDCT spectral energy of the 8 blocks is calculated and denoted as avgEner. There are two methods for calculating the average value of the MDCT spectral energy:

[0264] Method 1: Directly calculate the average value of the spectral energy of the 8 blocks of MDCT, that is, the average value of enerMdct[8].

[0265] Method 2: To reduce the impact of the block with the highest energy among the 8 blocks on the average calculation, the energy of the block with the highest energy can be removed before calculating the average.

[0266] The MDCT spectral energy of each block is compared with the average energy. If it is greater than a certain multiple of the average energy, the current block is considered to be a transient block (marked as 0). Otherwise, the current block is considered to be a non-transient block (marked as 1). All transient blocks form a transient group, and all non-transient blocks form a non-transient group.

[0267] For example, if the window type of the current frame is a short window, the initial grouping information obtained could be:

[0268] Number of groups: numGroups: 2.

[0269] Block index: 0 1 2 3 4 5 6 7.

[0270] Group indicator information: groupIndicator: 1 1 1 0 0 0 0 1.

[0271] The number of groups and group flag information need to be written into the bitstream and transmitted to the decoding end.

[0272] S39. Based on the grouping information, group and arrange the MDCT spectra of the M blocks to obtain the grouped MDCT spectra.

[0273] The specific scheme for grouping and arranging the MDCT spectra of the M blocks according to the grouping information can be any of the aforementioned steps S14 executed at the encoding end.

[0274] For example, some blocks belonging to the transient group out of the eight blocks in a short frame are placed at the beginning, and some blocks belonging to other groups are placed at the end.

[0275] Taking the example from step S38 as an example, if the grouping information is:

[0276] Block index: 0 1 2 3 4 5 6 7.

[0277] Group indicator information: groupIndicator: 1 1 1 0 0 0 0 1.

[0278] The resulting spectrum arrangement is as follows:

[0279] Block index: 3 4 5 6 0 1 2 7.

[0280] That is, the spectrum of the 0th block after the arrangement is the spectrum of the 3rd block before the arrangement, the spectrum of the 1st block after the arrangement is the spectrum of the 4th block before the arrangement, the spectrum of the 2nd block after the arrangement is the spectrum of the 5th block before the arrangement, the spectrum of the 3rd block after the arrangement is the spectrum of the 6th block before the arrangement, the spectrum of the 4th block after the arrangement is the spectrum of the 0th block before the arrangement, the spectrum of the 5th block after the arrangement is the spectrum of the 1st block before the arrangement, the spectrum of the 6th block after the arrangement is the spectrum of the 2nd block before the arrangement, and the spectrum of the 7th block after the arrangement is the spectrum of the 7th block before the arrangement.

[0281] S310. Perform intra-group spectrum interleaving on the grouped MDCT spectrum to obtain the intra-group interleaved MDCT spectrum.

[0282] After the MDCT spectra are grouped and arranged, interleaving processing is performed on each group. The processing method is similar to step S35, except that the interleaving processing is limited to the MDCT spectra belonging to the same group.

[0283] Taking the above example again, in the arranged spectrum, the transient group (the 3rd, 4th, 5th, and 6th blocks before arrangement, i.e. the 0th, 1st, 2nd, and 3rd blocks after arrangement) is interleaved, and the other groups (the 0th, 1st, 2nd, and 7th blocks before arrangement, i.e. the 4th, 5th, 6th, and 7th blocks after arrangement) are interleaved.

[0284] S311. Encode the MDCT spectrum after intragroup interleaving using an encoding neural network.

[0285] This application does not limit the specific method used to encode the intra-group interleaved MDCT spectrum using a coding neural network. For example, the intra-group interleaved MDCT spectrum is processed by a coding neural network to generate latent variables. The latent variables are then quantized to obtain quantized latent variables. Arithmetic coding is performed on the quantized latent variables, and the arithmetic coding result is written into the bitstream.

[0286] S312. If the current frame is not a short frame, then encode the MDCT spectrum of the current frame according to the encoding method corresponding to other frame types.

[0287] For encoding other types of frames, grouping, permutation, and intra-group interleaving can be omitted. For example, the MDCT spectrum of the current frame obtained in step S34 can be directly encoded using an encoding neural network.

[0288] For example, determine the window function corresponding to the window type, perform windowing processing on the audio signal of the current frame, and obtain the windowed signal; when the windows of adjacent frames overlap, perform a time-frequency forward transform on the windowed signal, such as MDCT transform, to obtain the MDCT spectrum of the current frame; and encode the MDCT spectrum of the current frame.

[0289] like Figure 9 As shown, the audio signal decoding method executed by the decoding end includes:

[0290] S41. Decode the received bitstream to obtain the window type of the current frame.

[0291] Determine whether the window type of the current frame is a short window. If yes, proceed to step S42; otherwise, proceed to step S410.

[0292] S42. If the window type of the current frame is a short window, decode the received bitstream to obtain the number of packets and packet flag information.

[0293] S43. Based on the received bitstream, decode it and use a decoding neural network to obtain the decoded MDCT spectrum.

[0294] Decoding neural networks correspond to encoding neural networks. For example, the specific method of decoding using a decoding neural network is as follows: Arithmetic decoding is performed on the received bitstream to obtain quantized latent variables. The quantized latent variables are then dequantized to obtain dequantized latent variables. These dequantized latent variables are then used as input and processed by the decoding neural network to generate the decoded MDCT spectrum.

[0295] S44. Based on the number of groups and group flag information, perform intra-group deinterleaving processing on the decoded MDCT spectrum to obtain the intra-group deinterleaved MDCT spectrum.

[0296] Based on the number of groups and the group indicator information, the MDCT spectrum blocks belonging to the same group are determined. For example, the decoded MDCT spectrum is divided into 8 blocks. The number of groups is equal to 2, and the group indicator information `groupIndicator` is 1 1 1 0 0 0 0 1. The number of bits with a flag value of 0 in the group indicator information is 4. Therefore, the MDCT spectra of the first 4 blocks in the decoded MDCT spectrum belong to one group, which is a transient group and requires intra-group deinterleaving processing. The number of bits with a flag value of 1 is 4. Therefore, the MDCT spectra of the last 4 blocks belong to one group, which is a non-transient group and requires intra-group deinterleaving processing. The MDCT spectrum of the 8 blocks obtained by intra-group deinterleaving processing is the MDCT spectrum of the 8 blocks after intra-group deinterleaving processing.

[0297] S45. Based on the number of groups and the grouping information, perform reverse grouping and rearrangement processing on the MDCT spectrum after deinterleaving within the group to obtain the MDCT spectrum after reverse grouping and rearrangement.

[0298] Based on the group indicator information groupIndicator, the MDCT spectra processed within the group are arranged into M blocks of spectra in chronological order.

[0299] For example, if the number of groups is 2 and the group indicator information `groupIndicator` is 1 1 1 0 0 0 0 1, then the MDCT spectrum of block 0 obtained from the intra-group deinterleaving process needs to be adjusted to the MDCT spectrum of block 3 (the element position index corresponding to the first bit with a value of 0 in the group indicator information is 3); the MDCT spectrum of block 1 obtained from the intra-group deinterleaving process needs to be adjusted to the MDCT spectrum of block 4 (the element position index corresponding to the second bit with a value of 0 in the group indicator information is 4); the MDCT spectrum of block 2 obtained from the intra-group deinterleaving process needs to be adjusted to the MDCT spectrum of block 5 (the element position index corresponding to the third bit with a value of 0 in the group indicator information is 5); and the MDCT spectrum of block 3 obtained from the intra-group deinterleaving process needs to be adjusted to the MDCT spectrum of block 6 (the element position index corresponding to the fourth bit with a value of 0 in the group indicator information is 5). The element position index corresponding to the bit in the group is 6); the MDCT spectrum of the 4th block obtained by the intra-group deinterleaving process is adjusted to the MDCT spectrum of the 0th block (the element position index corresponding to the first bit with a flag value of 1 in the group flag information is 0); the MDCT spectrum of the 5th block obtained by the intra-group deinterleaving process is adjusted to the MDCT spectrum of the 1st block (the element position index corresponding to the second bit with a flag value of 1 in the group flag information is 1); the MDCT spectrum of the 6th block obtained by the intra-group deinterleaving process is adjusted to the MDCT spectrum of the 2nd block (the element position index corresponding to the third bit with a flag value of 1 in the group flag information is 2); the MDCT spectrum of the 7th block obtained by the intra-group deinterleaving process is not adjusted and is directly used as the MDCT spectrum of the 7th block.

[0300] At the encoding end, the short frame spectrum after the spectrum is grouped and arranged is as follows: Block index 3 4 5 6 0 1 2 7.

[0301] At the decoding end, the short frame spectrum processed by reverse grouping is restored to 8 blocks of spectrum in chronological order: Block index 0 1 2 3 4 5 6 7.

[0302] S46. The MDCT spectrum processed by the inverse grouping arrangement is interleaved to obtain the interleaved MDCT spectrum.

[0303] If the window type of the current frame is a short window, the MDCT spectrum processed by reverse grouping is interleaved, using the same method as before.

[0304] S47. Perform decoding post-processing on the interleaved MDCT spectrum to obtain the decoded MDCT spectrum.

[0305] Post-decoding processing can include BWE inverse processing, TNS inverse processing, FDNS inverse processing, and so on.

[0306] S48. Perform deinterleaving on the MDCT spectrum after decoding to obtain the reconstructed MDCT spectrum.

[0307] S49. Perform inverse MDCT transform and windowing on the reconstructed MDCT spectrum to obtain the reconstructed audio signal.

[0308] The reconstructed MDCT spectrum consists of M blocks of MDCT spectra. An inverse MDCT transform is performed on the MDCT spectrum of each block. After windowing and aliasing addition of the inversely transformed signals, the reconstructed audio signal of a short frame can be obtained.

[0309] S410. If the window type of the current frame is another window type, decode it according to the decoding method corresponding to other types of frames to obtain the reconstructed audio signal.

[0310] For example, based on the received bitstream decoding, a decoding neural network is used to obtain the reconstructed MDCT spectrum. Inverse transform and OLA are then performed based on the window type (long window, in-window, out-window) to obtain the reconstructed audio signal.

[0311] Using the method proposed in this application, if the window type of the current frame is a short window, the number of groups and grouping flag information of the current frame are obtained based on the spectrum of the M blocks of the current frame; the spectrum of the M blocks of the current frame is grouped and arranged according to the number of groups and grouping flag information to obtain the grouped audio signal; and the grouped spectrum is encoded using an encoding neural network. This ensures that when the audio signal of the current frame is a transient signal, the MDCT spectrum containing transient features can be adjusted to a position with higher encoding importance, so that the audio signal reconstructed after neural network encoding and decoding can better preserve transient features.

[0312] The embodiments of this application can also be used for stereo coding, the difference being: First, the left and right channels of the stereo are processed according to steps S31-310 in the encoding end of the aforementioned embodiment to obtain the intra-group interleaved MDCT spectrum of the left channel and the intra-group interleaved MDCT spectrum of the right channel. Then, step S311 becomes: encoding the intra-group interleaved MDCT spectrum of the left channel and the intra-group interleaved MDCT spectrum of the right channel using an encoding neural network.

[0313] The input to the encoding neural network is no longer the mono MDCT spectrum after intra-channel interleaving, but the MDCT spectrum of the left channel and the MDCT spectrum of the right channel after intra-channel interleaving obtained by processing the left and right channels of the stereo according to steps S31-310.

[0314] The encoding neural network can be a CNN network, which takes the intra-group interleaved MDCT spectrum of the left channel and the intra-group interleaved MDCT spectrum of the right channel as inputs to the two channels of the CNN network.

[0315] Correspondingly, the decoding process includes:

[0316] Based on the received bitstream decoding, obtain the window type, number of groups, and group flag information of the left channel of the current frame.

[0317] Based on the received bitstream decoding, obtain the window type, number of groups, and group flag information of the right channel of the current frame.

[0318] Based on the received bitstream, the decoded stereo MDCT spectrum is obtained using a decoding neural network.

[0319] Based on the window type, number of groups, group flag information, and MDCT spectrum of the left channel of the current frame, the reconstructed left channel signal is obtained by following the steps of mono decoding on the decoding side in Embodiment 1.

[0320] Based on the window type, number of groups, group flag information, and MDCT spectrum of the right channel of the current frame, the reconstructed right channel signal is obtained by following the steps of mono decoding on the decoding side in Embodiment 1.

[0321] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0322] To facilitate better implementation of the above-described solutions in the embodiments of this application, related apparatus for implementing the above-described solutions is also provided below.

[0323] Please see Figure 10 As shown in the embodiment of this application, an audio encoding device 1000 may include: a transient identifier acquisition module 1001, a grouping information acquisition module 1002, a grouping arrangement module 1003, and an encoding module 1004, wherein...

[0324] A transient identifier acquisition module is used to obtain M transient identifiers of the M blocks based on the spectrum of the M blocks of the current frame of the audio signal to be encoded; the M blocks include a first block, and the transient identifier of the first block is used to indicate that the first block is a transient block or to indicate that the first block is a non-transient block;

[0325] The grouping information acquisition module is used to obtain the grouping information of the M blocks based on the M transient identifiers of the M blocks;

[0326] The grouping and arranging module is used to group and arrange the spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum to be encoded in the current frame;

[0327] The encoding module is used to encode the spectrum to be encoded using an encoding neural network to obtain a spectrum encoding result; and to write the spectrum encoding result into a bitstream.

[0328] Please see Figure 11 As shown in the figure, an audio decoding device 1100 provided in this application embodiment may include: a group information acquisition module 1101, a decoding module 1102, an inverse grouping module 1103, and an audio signal acquisition module 1104, wherein,

[0329] A grouping information acquisition module is used to obtain grouping information of M blocks of the current frame of the audio signal from the bitstream, wherein the grouping information is used to indicate M transient identifiers of the M blocks;

[0330] A decoding module is used to decode the bitstream using a decoding neural network to obtain the decoding spectrum of M blocks;

[0331] The inverse grouping module is used to perform inverse grouping and arranging processing on the decoded spectra of the M blocks according to the grouping information of the M blocks, so as to obtain the spectrum of the M blocks after inverse grouping processing;

[0332] The audio signal acquisition module is used to obtain the reconstructed audio signal of the current frame based on the spectrum of the inverse grouping process of the M blocks.

[0333] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0334] This application also provides a computer storage medium storing a program that performs some or all of the steps described in the above method embodiments.

[0335] The following describes another audio encoding device provided in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 12 As shown, the audio encoding device 1200 includes:

[0336] Receiver 1201, transmitter 1202, processor 1203, and memory 1204 (wherein the audio encoding device 1200 may contain one or more processors 1203). Figure 12(Taking a processor as an example). In some embodiments of this application, the receiver 1201, transmitter 1202, processor 1203, and memory 1204 can be connected via a bus or other means, wherein, Figure 12 Taking the example of a connection between China and Israel via a bus.

[0337] Memory 1204 may include read-only memory and random access memory, and provides instructions and data to processor 1203. A portion of memory 1204 may also include non-volatile random access memory (NVRAM). Memory 1204 stores operating system and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic business functions and handling hardware-based tasks.

[0338] Processor 1203 controls the operation of the audio encoding device; processor 1203 can also be called a central processing unit (CPU). In specific applications, the various components of the audio encoding device are coupled together through a bus system, which includes not only a data bus but also a power bus, control bus, and status signal bus, etc. However, for clarity, all buses are referred to as the bus system in the diagram.

[0339] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1203. The processor 1203 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 1203 or by instructions in the form of software. The processor 1203 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1204. Processor 1203 reads the information in memory 1204 and completes the steps of the above method in conjunction with its hardware.

[0340] The receiver 1201 can be used to receive input digital or character information and generate signal inputs related to the settings and function control of the audio encoding device. The transmitter 1202 may include a display device such as a display screen and can be used to output digital or character information through an external interface.

[0341] In this embodiment, processor 1203 is used to execute the aforementioned embodiments. Figure 3 , Figure 6 , Figure 8 The method shown is performed by the audio encoding device.

[0342] The following describes another audio decoding device provided in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 13 As shown, the audio decoding device 1300 includes:

[0343] Receiver 1301, transmitter 1302, processor 1303, and memory 1304 (wherein the audio decoding device 1300 may contain one or more processors 1303). Figure 13(Taking a processor as an example). In some embodiments of this application, the receiver 1301, transmitter 1302, processor 1303, and memory 1304 can be connected via a bus or other means, wherein... Figure 13 Taking the example of a connection between China and Israel via a bus.

[0344] Memory 1304 may include read-only memory and random access memory, and provides instructions and data to processor 1303. A portion of memory 1304 may also include NVRAM. Memory 1304 stores operating system and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic business functions and handling hardware-based tasks.

[0345] Processor 1303 controls the operation of the audio decoding device; processor 1303 can also be referred to as a CPU. In specific applications, the various components of the audio decoding device are coupled together through a bus system. This bus system includes not only a data bus but also a power bus, control bus, and status signal bus, etc. However, for clarity, all buses are referred to as the bus system in the diagram.

[0346] The methods disclosed in the embodiments of this application can be applied to processor 1303, or implemented by processor 1303. Processor 1303 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 1303 or by instructions in the form of software. The processor 1303 can be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1304, and processor 1303 reads the information in memory 1304 and completes the steps of the above method in combination with its hardware.

[0347] In this embodiment, processor 1303 is used to execute the aforementioned embodiments. Figure 4 , Figure 7 , Figure 9The method shown is performed by the audio decoding device.

[0348] In another possible design, when the audio encoding or decoding device is a chip within the terminal, the chip includes a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, pins, or circuitry. The processing unit can execute computer-executable instructions stored in the storage unit to cause the chip within the terminal to execute the audio encoding method of any of the first aspects or the audio decoding method of any of the second aspects described above. Optionally, the storage unit can be a storage unit within the chip, such as a register or cache. Alternatively, the storage unit can be a storage unit located outside the chip within the terminal, such as read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0349] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of programs in the first or second aspect of the above methods.

[0350] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0351] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0352] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0353] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

Claims

1. A method for encoding audio signals, characterized in that, include: The M transient identifiers of the M blocks are obtained based on the spectrum of the M blocks of the current frame of the audio signal to be encoded; the M blocks include a first block, and the transient identifier of the first block is used to indicate that the first block is a transient block or to indicate that the first block is a non-transient block; The grouping information of the M blocks is obtained based on the M transient identifiers of the M blocks; The spectra of the M blocks are grouped and arranged according to the grouping information of the M blocks to obtain the spectrum to be encoded in the current frame. The grouping and arrangement is used to adjust the order of the spectra of the M blocks in the spectrum to be encoded in the current frame. The spectrum to be encoded is encoded using an encoding neural network to obtain a spectrum encoding result; The spectrum encoding result is written into the bitstream.

2. The method according to claim 1, characterized in that, The method further includes: The grouping information of the M blocks is encoded to obtain the grouping information encoding result; The encoded result of the grouping information is written into the bitstream.

3. The method according to claim 1 or 2, characterized in that, The grouping information of the M blocks includes: the number of groups or the grouping number identifier of the M blocks, wherein the grouping number identifier is used to indicate the number of groups; when the number of groups is greater than 1, the grouping information of the M blocks further includes: M transient identifiers of the M blocks; or, the grouping information of the M blocks includes: M transient identifiers of the M blocks.

4. The method according to any one of claims 1 to 3, characterized in that, The step of grouping and arranging the spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum to be encoded in the current frame includes: The spectra of the M blocks that are indicated as transient blocks by the M transient identifiers are assigned to the transient group, and the spectra of the M blocks that are indicated as non-transient blocks by the M transient identifiers are assigned to the non-transient group; The spectrum of the blocks in the transient group is arranged before the spectrum of the blocks in the non-transient group to obtain the spectrum to be encoded for the current frame.

5. The method according to any one of claims 1 to 3, characterized in that, The step of grouping and arranging the spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum to be encoded in the current frame includes: The spectra of the M blocks that are indicated as transient blocks by the M transient identifiers are arranged before the spectra of the M blocks that are indicated as non-transient blocks by the M transient identifiers, so as to obtain the spectrum to be encoded in the current frame.

6. The method according to any one of claims 1 to 5, characterized in that, Before encoding the spectrum to be encoded using a coding neural network, the method further includes: The spectrum to be encoded is subjected to intra-group interleaving to obtain the spectrum of M blocks with intra-group interleaving. The process of encoding the spectrum to be encoded using a coding neural network includes: The spectra of the M blocks in the intra-group interleaving process are encoded using an encoding neural network.

7. The method according to claim 6, characterized in that, The number of blocks identified as transient by the M transient identifiers is P, and the number of blocks identified as non-transient by the M transient identifiers is Q, where M = P + Q; The intra-group interleaving process for the spectrum to be encoded includes: The spectra of the P blocks are interleaved to obtain the interleaved spectra of the P blocks; The spectra of the Q blocks are interleaved to obtain the interleaved spectra of the Q blocks; The process of encoding the spectra of the M blocks undergoing intra-group interleaving using a coding neural network includes: The spectrum of the interleaving process of the P blocks and the spectrum of the interleaving process of the Q blocks are encoded using an encoding neural network.

8. The method according to any one of claims 1 to 7, characterized in that, Before obtaining the M transient identifiers of the M blocks based on the spectra of the M blocks of the current frame of the audio signal to be encoded, the method further includes: Obtain the window type of the current frame, wherein the window type is a short window type or a non-short window type; The step of obtaining the M transient identifiers of the M blocks based on the spectrum of the M blocks of the current frame of the audio signal to be encoded is performed only when the window type is a short window type.

9. The method according to claim 8, characterized in that, The method further includes: The window type is encoded to obtain the window type encoding result; The window type encoding result is written into the bitstream.

10. The method according to any one of claims 1 to 9, characterized in that, The step of obtaining the M transient identifiers of the M blocks based on the spectra of the M blocks of the current frame of the audio signal to be encoded includes: The M spectral energies of the M blocks are obtained based on their spectra. The average spectral energy of the M blocks is obtained based on the M spectral energies; The M transient identifiers of the M blocks are obtained based on the M spectral energies and the average spectral energy.

11. The method according to claim 10, characterized in that, When the spectral energy of the first block is greater than K times the average spectral energy, the transient flag of the first block indicates that the first block is a transient block; or, When the spectral energy of the first block is less than or equal to K times the average value of the spectral energy, the transient flag of the first block indicates that the first block is a non-transient block; Wherein, K is a real number greater than or equal to 1.

12. A method for decoding an audio signal, characterized in that, include: Obtain the grouping information of M blocks of the current frame of the audio signal from the bitstream, wherein the grouping information is used to indicate the M transient identifiers of the M blocks; The bitstream is decoded using a decoding neural network to obtain the decoded spectra of the M blocks; The decoded spectra of the M blocks are reverse-grouped according to the grouping information of the M blocks to obtain the spectrum of the reverse-grouped M blocks. The reverse-grouping process is used to recover the order of the decoded spectra of the M blocks in the spectrum of the current frame. The reconstructed audio signal of the current frame is obtained based on the spectrum of the M blocks after inverse grouping and arrangement processing.

13. The method according to claim 12, characterized in that, Before performing the reverse grouping and rearranging process on the decoded spectra of the M blocks based on their grouping information, the method further includes: The decoded spectra of the M blocks are subjected to intra-group deinterleaving processing to obtain the intra-group deinterleaving processed spectra of the M blocks; The step of performing reverse grouping and rearranging of the decoded spectra of the M blocks according to the grouping information of the M blocks includes: The inverse grouping arrangement process is performed on the spectrum of the intra-group deinterleaving process of the M blocks based on the grouping information of the M blocks.

14. The method according to claim 13, characterized in that, The number of blocks identified as transient by the M transient identifiers is P, and the number of blocks identified as non-transient by the M transient identifiers is Q, where M = P + Q; The intra-group deinterleaving process for the decoded spectra of the M blocks includes: The decoded spectra of the P blocks are deinterleaved; and, The decoding spectra of the Q blocks are deinterleaved.

15. The method according to any one of claims 12 to 14, characterized in that, The number of blocks identified as transient by the M transient identifiers is P, and the number of blocks identified as non-transient by the M transient identifiers is Q, where M = P + Q; The step of performing reverse grouping and rearranging of the decoded spectra of the M blocks according to the grouping information of the M blocks includes: The indices of the P blocks are obtained based on the grouping information of the M blocks; The indices of the Q blocks are obtained based on the grouping information of the M blocks; The decoding spectrum of the M blocks is subjected to the reverse grouping and permutation process based on the indices of the P blocks and the indices of the Q blocks.

16. The method according to any one of claims 12 to 15, characterized in that, The method further includes: The window type of the current frame is obtained from the bitstream, wherein the window type is either a short window type or a non-short window type; The step of obtaining the grouping information of the M blocks of the current frame from the bitstream is only performed when the window type of the current frame is a short window type.

17. The method according to any one of claims 12 to 16, characterized in that, The grouping information of the M blocks includes: the number of groups or the group number identifier of the M blocks, wherein the group number identifier is used to indicate the number of groups; when the number of groups is greater than 1, the grouping information of the M blocks also includes: the M transient identifiers of the M blocks; or, The grouping information of the M blocks includes: M transient identifiers of the M blocks.

18. An audio signal encoding device, characterized in that, include: A transient identifier acquisition module is used to obtain M transient identifiers of the M blocks based on the spectrum of the M blocks of the current frame of the audio signal to be encoded; the M blocks include a first block, and the transient identifier of the first block is used to indicate that the first block is a transient block or to indicate that the first block is a non-transient block; The grouping information acquisition module is used to obtain the grouping information of the M blocks based on the M transient identifiers of the M blocks; The grouping and arranging module is used to group and arrange the spectra of the M blocks according to the grouping information of the M blocks to obtain the spectrum to be encoded. The grouping and arranging is used to adjust the arrangement order of the spectra of the M blocks in the spectrum to be encoded in the current frame. The encoding module is used to encode the spectrum to be encoded using an encoding neural network to obtain a spectrum encoding result; and to write the spectrum encoding result into a bitstream.

19. An audio signal decoding device, characterized in that, include: A grouping information acquisition module is used to obtain grouping information of M blocks of the current frame of the audio signal from the bitstream, wherein the grouping information is used to indicate M transient identifiers of the M blocks; A decoding module is used to decode the bitstream using a decoding neural network to obtain the decoding spectrum of M blocks; The reverse grouping and arrangement module is used to perform reverse grouping and arrangement processing on the decoded spectrum of the M blocks according to the grouping information of the M blocks, so as to obtain the spectrum of the reverse grouping and arrangement processing of the M blocks. The reverse grouping and arrangement processing is used to recover the arrangement order of the decoded spectrum of the M blocks in the spectrum of the current frame. An audio signal acquisition module is used to obtain a reconstructed audio signal based on the spectrum of the M blocks in reverse grouping and arrangement processing.

20. An audio signal encoding device, characterized in that, The audio signal encoding device includes at least one processor, the at least one processor being coupled to a memory to read and execute instructions in the memory to implement the method as described in any one of claims 1 to 11.

21. The audio signal encoding apparatus according to claim 20, characterized in that, The audio signal encoding device further includes the memory.

22. An audio signal decoding device, characterized in that, The audio signal decoding device includes at least one processor, the at least one processor being coupled to a memory to read and execute instructions in the memory to implement the method as described in any one of claims 12 to 17.

23. The audio signal decoding apparatus according to claim 22, characterized in that, The audio signal decoding device further includes the memory.

24. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 11 or 12 to 17.

25. A computer-readable storage medium comprising a bitstream generated by the method as described in any one of claims 1 to 11.

Images