Encoding and decoding methods and electronic devices

Abstract

This application provides an encoding / decoding method and an electronic device. The encoding method includes: first, acquiring data to be encoded; then, performing feature transformation on the data to be encoded according to a preset network to obtain the latent features corresponding to the data to be encoded; subsequently, performing wavelet decomposition on the latent features to obtain N wavelet coefficient groups, where N is an integer greater than 1; and then performing vector quantization on each of the N wavelet coefficient groups to obtain the bitstream of the data to be encoded. Compared to directly performing vector quantization on the latent features, decomposing the latent features into multiple smaller wavelet coefficient groups before performing vector quantization results in a smaller codebook size for vector quantization at the same bit rate, thereby reducing the storage space occupied by the codebook.

Description

Technical Field

[0001] This application relates to the field of encoding and decoding, and more particularly to an encoding and decoding method and an electronic device. Background Technology

[0002] With the continuous development and maturation of artificial intelligence (AI) technology, more and more fields have begun to use neural network technology to replace or combine with traditional technologies (for example, introducing AI technology into the field of audio and video encoding and decoding), and have made great progress.

[0003] Currently, the existing technology for encoding audio data involves directly performing vector quantization on the audio data after it has passed through an AI encoding network to obtain a bitstream. However, when the vector obtained after passing through the AI ​​encoding network is long, the required codebook size also increases; correspondingly, the codebook occupies more storage space. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides an encoding / decoding method and an electronic device. This method can reduce the storage space occupied by codebooks.

[0005] In a first aspect, embodiments of this application provide an encoding method, which includes: first, acquiring data to be encoded; then, performing feature transformation on the data to be encoded according to a preset network to obtain latent features corresponding to the data to be encoded; subsequently, performing wavelet decomposition on the latent features to obtain N wavelet coefficient groups, where N is an integer greater than 1; and then performing vector quantization on each of the N wavelet coefficient groups to obtain the bitstream of the data to be encoded. Compared to directly performing vector quantization on the latent features, decomposing the latent features into multiple wavelet coefficient groups of smaller size before performing vector quantization results in a smaller codebook size used for vector quantization at the same bitrate, thereby reducing the storage space occupied by the codebook in the encoding end.

[0006] For example, assuming the bitstream is 16 * 50 = 800 bps, meaning 50 frames of audio data can be transmitted per second, transmitting one frame requires 16 bits. If the latent feature corresponding to one frame of audio data is a 1 * 16 vector, then the codebook size required for vector quantization is 16 * 2. 16 If the latent features corresponding to a frame of audio data are decomposed using wavelet decomposition to obtain four wavelet coefficient groups, and each wavelet coefficient group is a 1*4 vector, then the codebook size required for vector quantization is 4*2. 4 Therefore, it can be seen that by decomposing a large matrix into multiple smaller matrices before performing vector quantization, the required codebook size is greatly reduced.

[0007] For example, the data to be encoded can be source data, which can refer to the data collected by the acquisition module. Source data can include audio data, video data, or images.

[0008] For example, audio data includes, but is not limited to, mono audio data, stereo audio data, multi-channel audio data, or 3D (3D) audio data, etc., and this application does not limit this. Video data may include 2D (2D) video data, 3D video data, etc., and this application does not limit this. Images may include 2D images, 3D images, etc., and this application does not limit this.

[0009] For example, the data to be encoded can be frequency domain data obtained by frequency domain transformation of the source data, hereinafter referred to as the frequency domain data of the source data.

[0010] For example, the data to be encoded may be the data after preprocessing the source data / frequency domain data of the source data; wherein, the preprocessing includes but is not limited to framing, windowing, etc., and this application does not limit it.

[0011] For example, the preset network is an AI encoding network, which can be implemented by a neural network.

[0012] For example, the dimensions of any two wavelet coefficient groups in the N wavelet coefficient groups can be the same or different, and this application does not impose any restrictions on this.

[0013] For example, the code stream includes N wavelet coefficient groups corresponding to vector quantization value indices in the codebook.

[0014] According to the first aspect, wavelet decomposition is performed on the latent features to obtain N wavelet coefficient sets, including: multi-level wavelet decomposition of the latent features to obtain N wavelet coefficient sets. In this way, the size of the wavelet coefficient sets can be further reduced, thereby reducing the size of the codebook required for vector quantization, and thus further reducing the storage space occupied by the codebook at the encoding end.

[0015] According to the first aspect, or any implementation of the first aspect above, vector quantization is performed on the N wavelet coefficient groups respectively to obtain the bitstream of the data to be encoded, including: determining the target codebooks corresponding to the N wavelet coefficient groups respectively from the M codebooks that have been trained; where M is a positive integer less than or equal to N; and performing vector quantization on the N wavelet coefficient groups respectively based on the target codebooks corresponding to the N wavelet coefficient groups to obtain the bitstream of the data to be encoded.

[0016] According to the first aspect, or any implementation of the first aspect above, the target codebooks corresponding to the N wavelet coefficient groups are determined from the trained M codebooks, including: determining the target codebooks corresponding to the N wavelet coefficient groups from the M codebooks according to a first preset relationship; wherein the first preset relationship is used to describe the relationship between the M codebooks and the N wavelet coefficient groups.

[0017] For example, when M equals N, one codebook corresponds to one wavelet coefficient set; when M is less than N, one codebook corresponds to multiple wavelet coefficient sets.

[0018] According to the first aspect, or any implementation of the first aspect above, from the trained M-group codebooks, determine the target codebooks corresponding to the N wavelet coefficient groups respectively, including: for the i-th wavelet coefficient group among the N wavelet coefficient groups, where i is an integer between 1 and N; determine the M-group quantization error based on the i-th wavelet coefficient group and the M-group vector quantization value corresponding to the i-th wavelet coefficient group, wherein the M-group vector quantization value is obtained by vector quantizing the i-th wavelet coefficient group based on the M-group codebooks respectively; determine the codebook corresponding to the minimum vector quantization error among the M-group vector quantization errors; and determine the codebook corresponding to the minimum vector quantization error as the target codebook corresponding to the i-th wavelet coefficient group. This can reduce coding error and improve coding quality (i.e., data reconstruction quality).

[0019] Depending on the first aspect, or any of the implementations of the first aspect above, the precision of some codebooks in the M-group codebooks differs.

[0020] Since different wavelet coefficient sets have varying degrees of importance to data reconstruction quality, when designing codebooks for different wavelet coefficient sets, the precision (i.e., the number of codewords in the codebook) of the codebooks corresponding to wavelet coefficient sets with high importance to data reconstruction quality can be increased, while the precision of the codebooks corresponding to wavelet coefficient sets with low importance to data reconstruction quality can be decreased. Thus, by using different codebooks for vector quantization for different wavelet coefficient sets, the proportion of wavelet coefficient sets with high importance to data reconstruction quality in the bitstream can be increased at the same bitrate, thereby improving the data reconstruction quality.

[0021] For example, the precision of all codebooks in the M codebooks is different.

[0022] For example, in the M codebooks, some codebooks have different precision, while others have the same precision.

[0023] According to the first aspect, or any implementation of the first aspect above, the method further includes: performing vector splitting on X1 wavelet coefficient groups out of N wavelet coefficient groups to obtain U1 sub-vectors corresponding to each of the X1 wavelet coefficient groups, where X1 is a positive integer less than or equal to N, and U1 is an integer greater than 1; performing vector quantization on each of the N wavelet coefficient groups to obtain the bitstream of the data to be encoded, including: performing vector quantization on the remaining X2 wavelet coefficient groups out of the N wavelet coefficient groups, and performing vector quantization on the U1 sub-vectors corresponding to each of the X1 wavelet coefficient groups to obtain the bitstream of the data to be encoded; where X2 is an integer less than N, and X1 + X2 = N. In this way, the wavelet coefficient group vectors can be split into smaller sub-vectors, which can further reduce the size of the codebook required for vector quantization, thereby further reducing the storage space occupied by the codebook at the encoding and decoding end.

[0024] According to the first aspect, or any implementation of the first aspect above, vector quantization is performed on the additional X2 wavelet coefficient groups from the N wavelet coefficient groups, and vector quantization is performed on the U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively, to obtain the bitstream of the data to be encoded. This includes: selecting U2 target sub-vectors to be vector quantized from the U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively, and selecting X3 wavelet coefficient groups to be vector quantized from the X2 wavelet coefficient groups, where X3 is a positive integer less than or equal to X2, and U2 is a positive integer; performing vector quantization on the X3 wavelet coefficient groups and on the U2 target sub-vectors to obtain the bitstream of the data to be encoded. In this way, by discarding some sub-vectors and / or some wavelet coefficient groups, multi-rate transmission can be achieved without training multiple sets of preset networks.

[0025] For example, based on the target bit rate, selecting U2 target sub-vectors to be vector quantized from the U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively includes: selecting U2 target sub-vectors from the U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively based on the target bit rate and a third preset relationship; wherein, the third preset relationship is used to describe the relationship between multiple bit rates and multiple sub-vectors.

[0026] For example, based on the target bit rate, selecting U2 target sub-vectors to be vector quantized from the U1 sub-vectors corresponding to the X1 wavelet coefficient groups includes: determining the numerical feature information corresponding to each of the X1 wavelet coefficient groups, where the numerical feature information includes at least one of the following: frequency band, energy, variance, or mean; and selecting U2 target sub-vectors from the U1 sub-vectors corresponding to each of the X1 wavelet coefficient groups based on the target bit rate and the numerical feature information. The numerical feature information is related to the data reconstruction quality. By selecting the target sub-vectors to be vector quantized according to their importance to the data reconstruction quality—for example, selecting sub-vectors with high importance to the data reconstruction quality as the target sub-vectors to be vector quantized—it is possible to ensure data reconstruction quality as much as possible while achieving multi-bit rate transmission.

[0027] According to the first aspect, or any implementation of the first aspect above, vector quantization is performed on N wavelet coefficient groups to obtain the bitstream of the data to be encoded, including: selecting G wavelet coefficient groups to be vector quantized from the N wavelet coefficient groups according to the target bit rate, where G is a positive integer less than N; and performing vector quantization on the G wavelet coefficient groups to obtain the bitstream of the data to be encoded. In this way, by discarding some wavelet coefficient groups, multi-bitrate transmission can be achieved without training multiple pre-set networks.

[0028] According to the first aspect, or any implementation of the first aspect above, select G wavelet coefficient groups to be vector quantized from N wavelet coefficient groups, including: determining G wavelet coefficient groups based on the target code rate and a second preset relationship; wherein, the second preset relationship is used to describe the relationship between multiple code rates and N wavelet coefficient groups.

[0029] According to the first aspect, or any implementation thereof, based on the target bit rate, G wavelet coefficient groups to be vector-quantized are selected from N wavelet coefficient groups. This includes: determining the numerical feature information corresponding to each of the N wavelet coefficient groups, where the numerical feature information includes at least one of the following: frequency band, energy, variance, or mean; and selecting G wavelet coefficient groups to be vector-quantized from the N wavelet coefficient groups based on the target bit rate and the numerical feature information corresponding to each of the N wavelet coefficient groups. The numerical feature information is related to the data reconstruction quality. By selecting the wavelet coefficient groups to be vector-quantized according to their importance to the data reconstruction quality—for example, selecting wavelet coefficient groups with high importance to the data reconstruction quality as the wavelet coefficient groups to be vector-quantized—the data reconstruction quality can be guaranteed as much as possible while achieving multi-bit rate transmission.

[0030] According to the first aspect, or any implementation of the first aspect above, the code stream includes: code book identifier.

[0031] For example, a codebook identifier can be a codebook index.

[0032] According to the first aspect, or any implementation of the first aspect above, the bitstream includes: wavelet coefficient group identifier.

[0033] For example, the wavelet coefficient group identifier can be a wavelet coefficient group identifier.

[0034] For example, the bitstream includes: subvector identifiers of the target subvector.

[0035] For example, a subvector identifier can be a subvector index.

[0036] Secondly, embodiments of this application provide a decoding method, which includes: first, acquiring a bitstream to be decoded; then, performing vector inverse quantization on the bitstream to obtain N wavelet coefficient groups, where N is an integer greater than 1; subsequently, performing wavelet synthesis on the N wavelet coefficient groups to obtain latent features; and then performing feature transformation on the latent features according to a preset network to obtain reconstructed data. Thus, compared to performing vector inverse quantization on a bitstream obtained from large matrix vector quantization, the decoding end performs vector inverse quantization on multiple bitstreams obtained from small matrix vector quantization, requiring a smaller codebook size, thereby reducing the storage space occupied by the codebook at the decoding end.

[0037] According to the second aspect, wavelet synthesis is performed on N wavelet coefficient groups to obtain latent features, including: multi-level wavelet synthesis is performed on N wavelet coefficient groups to obtain latent features.

[0038] According to the second aspect, or any implementation of the second aspect above, vector inverse quantization is performed based on the code stream to obtain N wavelet coefficient groups, including: determining the target codebooks corresponding to the N wavelet coefficient groups from the M trained codebooks; where M is a positive integer less than or equal to N; and performing vector inverse quantization based on the target codebooks and code streams corresponding to the N wavelet coefficient groups to obtain N wavelet coefficient groups.

[0039] According to the second aspect, or any implementation of the second aspect above, the target codebooks corresponding to the N wavelet coefficient groups are determined from the trained M codebooks, including: determining the target codebooks corresponding to the N wavelet coefficient groups from the M codebooks according to the first preset relationship; wherein, the first preset relationship includes the relationship between the M codebooks and the N wavelet coefficient groups.

[0040] According to the second aspect, or any implementation of the second aspect above, the bitstream includes a codebook identifier;

[0041] From the trained M sets of codebooks, determine the target codebooks corresponding to the N wavelet coefficient groups respectively, including: determining the target codebooks corresponding to the N wavelet coefficient groups respectively based on the codebook identifiers in the code stream.

[0042] According to the second aspect, or any two implementations of the second aspect above, the precision of some codebooks in the M codebooks is different.

[0043] According to the second aspect, or any implementation of the second aspect above, vector dequantization is performed based on the code stream to obtain N wavelet coefficient groups, including: performing vector dequantization based on the code stream to obtain U1 sub-vectors and X2 wavelet coefficient groups corresponding to X1 wavelet coefficient groups respectively, where X1 is a positive integer less than or equal to N, X2 is an integer less than N, X1+X2=N, and U1 is an integer greater than 1; vector merging is performed on the U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively to obtain X1 wavelet coefficient groups.

[0044] According to the second aspect, or any implementation of the second aspect above, the code stream includes wavelet coefficient group identifiers; vector inverse quantization is performed based on the code stream to obtain N wavelet coefficient groups, including: vector inverse quantization is performed based on the code stream to obtain G wavelet coefficient groups, where G is a positive integer less than N; wavelet synthesis is performed on the N wavelet coefficient groups to obtain latent features, including: wavelet synthesis is performed on the G wavelet coefficient groups according to the wavelet coefficient group identifiers to obtain latent features.

[0045] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0046] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the electronic device to perform the encoding method in the first aspect or any possible implementation of the first aspect.

[0047] The third aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the third aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0048] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the electronic device to perform the decoding method in the second aspect or any possible implementation of the second aspect.

[0049] The fourth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects of the fourth aspect and any implementation thereof can be found in the technical effects of the second aspect and any implementation thereof, as described above, and will not be repeated here.

[0050] Fifthly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to execute the encoding method in the first aspect or any possible implementation of the first aspect.

[0051] The fifth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the fifth aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0052] In a sixth aspect, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to execute the decoding method in the second aspect or any possible implementation of the second aspect.

[0053] The sixth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects of the sixth aspect and any implementation thereof are similar to those of the second aspect and any implementation thereof, and will not be repeated here.

[0054] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to execute the encoding method in the first aspect or any possible implementation thereof.

[0055] The seventh aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the seventh aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0056] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to execute the decoding method in the second aspect or any possible implementation thereof.

[0057] The eighth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the eighth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0058] Ninthly, embodiments of this application provide a computer program product, which includes a software program that, when executed by a computer or processor, causes the computer or processor to perform the encoding method in the first aspect or any possible implementation thereof.

[0059] The ninth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the ninth aspect and any implementation thereof are similar to those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0060] In a tenth aspect, embodiments of this application provide a computer program product, which includes a software program that, when executed by a computer or processor, causes the computer or processor to perform the decoding method in the second aspect or any possible implementation thereof.

[0061] The tenth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the tenth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here. Attached Figure Description

[0062] Figure 1a A schematic diagram of the system framework is shown;

[0063] Figure 1bA schematic diagram of the system framework is shown;

[0064] Figure 1c This is a schematic diagram illustrating an application scenario;

[0065] Figure 1d This is a schematic diagram illustrating an application scenario;

[0066] Figure 2 This is a schematic diagram illustrating the encoding process as an example.

[0067] Figure 3 This is a schematic diagram illustrating the decoding process as an example.

[0068] Figure 4a This is an example of wavelet decomposition.

[0069] Figure 4b This is an example of wavelet decomposition.

[0070] Figure 5 This is a schematic diagram illustrating the encoding process as an example.

[0071] Figure 6 This is a schematic diagram illustrating the decoding process as an example.

[0072] Figure 7 This is a schematic diagram illustrating the encoding process as an example.

[0073] Figure 8 This is a schematic diagram illustrating the decoding process as an example.

[0074] Figure 9 This is a schematic diagram illustrating the encoding process as an example.

[0075] Figure 10 This is a schematic diagram illustrating the decoding process as an example.

[0076] Figure 11 This is a schematic diagram illustrating the encoding process as an example.

[0077] Figure 12 This is a schematic diagram illustrating the decoding process as an example.

[0078] Figure 13 This is a schematic diagram of the structure of an exemplary device. Detailed Implementation

[0079] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0080] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0081] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0082] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0083] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0084] Figure 1a A schematic diagram of the system framework is shown.

[0085] Reference Figure 1a For example, the system of this application may include a first terminal, a second terminal, and wired or wireless network communication devices. The first and second terminals may include, but are not limited to: mobile phones, personal computers (PCs), laptops, tablets, smart TVs, mobile internet devices (MIDs), wearable devices (such as smartwatches, smart glasses, or smart helmets), virtual reality (VR) devices, augmented reality (AR) devices, etc., and this application does not impose any limitations on these. Wired network communication devices may include routers, switches, etc., and wireless network communication devices may include base stations, core networks, wireless network cards, wireless access points (APs), etc., and this application does not impose any limitations on these.

[0086] Reference Figure 1aFor example, the first terminal may include: a data acquisition module, an encoder, a channel coding module, a data playback module, a decoder, and a channel decoding module. The second terminal may also include: a data acquisition module, an encoder, a channel coding module, a data playback module, a decoder, and a channel decoding module. It should be understood that... Figure 1a The first and second terminals shown are merely examples, and the first and second terminals may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 1a The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits. Furthermore, the modules contained in the first terminal and the second terminal may be the same or different, and this application does not impose any limitations on this.

[0087] For example, the data acquisition module of the first terminal can acquire media data (such as audio data, video data, or images) and send the media data to the encoder. Next, the encoder can perform source encoding on the media data to obtain a bitstream and send the bitstream to the channel coding module. Subsequently, the channel coding module performs channel coding on the bitstream to obtain data that can be used for transmission; then, the data that can be used for transmission is sent to a wired or wireless network communication device connected to the first terminal via a wired or wireless network communication device; finally, the wired or wireless network communication device connected to the second terminal sends the data that can be used for transmission to the second terminal.

[0088] For example, the channel decoding module of the second terminal can perform channel decoding on the received data to obtain a bitstream and send the bitstream to the decoder. Then, the decoder can perform source decoding on the bitstream to obtain media data; and then input the decoded media data to the data playback module for playback.

[0089] It should be understood that the process of the second terminal encoding the collected media data into a bitstream and sending it to the first terminal, and the first terminal performing source decoding on the bitstream and playing the decoded media data, is similar to the process described above and will not be repeated here.

[0090] It should be noted that audio data includes, but is not limited to, mono audio data, stereo audio data, multi-channel audio data, or 3D (3D) audio data, etc., and this application does not impose any limitations on this. Video data may include 2D (2D) video data, 3D video data, etc., and this application does not impose any limitations on this. Images may include 2D images, 3D images, etc., and this application does not impose any limitations on this.

[0091] In this way, the two terminals can encode and decode media data through their internal codecs, enabling the interaction of media data.

[0092] Figure 1b A schematic diagram of the system framework is shown.

[0093] Reference Figure 1b In one possible embodiment, the system of this application may include a first terminal, a second terminal, and a transcoding device. The transcoding device may include devices with transcoding capabilities, such as wireless access networks, core network equipment, media resource servers, etc., and this application does not impose any limitations on this.

[0094] For example, Figure 1b The first terminal in the middle and Figure 1a Similar to the first terminal in the middle, Figure 1b The second terminal in Figure 1a The second terminal is similar and will not be described in detail here. Figure 1b The codecs in the first terminal and the codecs in the second terminal support different encoding protocols. To facilitate differentiation, they can be... Figure 1b The encoder of the first terminal is called the first encoder. Figure 1b The decoder of the first terminal is called the first decoder, which will Figure 1b The encoder of the second terminal is called the second encoder. Figure 1b The decoder of the second terminal is called the second decoder.

[0095] Reference Figure 1b For example, a transcoding device may include a first decoder, a first encoder, a second decoder, a second encoder, a channel coding module, and a channel decoding module. It should be understood that... Figure 1b The transcoding device shown is only an example, and the transcoding device may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. Figure 1b The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0096] Reference Figure 1b (1) For example, the first terminal performs channel coding on the first bitstream obtained by the first encoder encoding the media data, and then sends it to the transcoding device; this is consistent with... Figure 1a The process described is similar and will not be repeated here.

[0097] Reference Figure 1b(1) For example, the channel decoding module of the transcoding device can perform channel decoding on the data received from the first terminal to obtain a first bitstream and send the first bitstream to the first decoder; the first decoder performs source decoding on the first bitstream to obtain media data. Then the first decoder sends the decoded media data to the second encoder, which performs source encoding to obtain a second bitstream and sends the second bitstream to the channel encoding module for channel encoding to obtain data that can be used for transmission and sends the data that can be used for transmission to the second terminal.

[0098] Reference Figure 1b (1) For example, the second terminal performs channel decoding on the received data to obtain a second bitstream, and then uses a second decoder to perform source decoding on the second bitstream to obtain media data; then the media data can be played. This is similar to... Figure 1a The process described is similar and will not be repeated here.

[0099] Reference Figure 1b (2) For example, the second terminal performs channel coding on the third bitstream obtained by encoding the media data by the second encoder and then sends it to the transcoding device; this is consistent with... Figure 1a The process described is similar and will not be repeated here.

[0100] Reference Figure 1b (2) For example, the channel decoding module of the transcoding device can perform channel decoding on the data received from the second terminal to obtain a third bitstream and send the third bitstream to the second decoder; the second decoder performs source decoding on the third bitstream to obtain media data. Then the second decoder sends the decoded media data to the first encoder, which performs source encoding to obtain a fourth bitstream and sends the fourth bitstream to the channel encoding module for channel encoding to obtain data that can be used for transmission and sends the data that can be used for transmission to the first terminal.

[0101] Reference Figure 1b (2) For example, the first terminal performs channel decoding on the received data to obtain a fourth bitstream, and then uses a first decoder to perform source decoding on the fourth bitstream to obtain media data; then the media data can be played. This is similar to... Figure 1a The process described is similar and will not be repeated here.

[0102] In this way, two terminals that support different encoding protocols can encode media data through the codec of the transcoding device, thus enabling the exchange of media data.

[0103] Figure 1c This is a schematic diagram illustrating an exemplary application scenario. Wherein, in Figure 1c In this embodiment, the application scenario is a live streaming scenario, and the media data is 3D audio data.

[0104] Reference Figure 1c For example, the live streaming terminal inputs the audio data collected during the live broadcast into the 3D audio production module. The 3D audio production module then creates 3D audio data for the live broadcast and sends it to the 3D audio encoder. The 3D audio encoder then encodes the 3D audio data to obtain the corresponding bitstream, which is then sent to the set-top box via the broadcast television network. Subsequently, the 3D audio decoder in the set-top box decodes the received bitstream to obtain the 3D audio data for the live broadcast, and then sends it to the television speakers for playback.

[0105] Figure 1d This is a schematic diagram illustrating an exemplary application scenario. Wherein, in Figure 1d In this embodiment, the application scenario is the post-production scenario of audio, and the media data is 3D audio data.

[0106] Reference Figure 1d For example, the 3D audio production terminal inputs the audio data of the recorded program into the 3D audio production module, which then processes the acquired audio data into 3D audio data for the recorded program and sends it to the 3D audio encoder. The 3D audio encoder then encodes the 3D audio data to obtain the corresponding bitstream. Alternatively, the 3D audio production terminal can send the bitstream to a network receiver via the internet. The 3D audio decoder in the network receiver can then decode the received bitstream to obtain the 3D audio data for the recorded program, and then send it to the television speakers for playback. Conversely, the 3D audio production terminal can also send the bitstream to a mobile terminal via the internet. The 3D audio decoder in the mobile terminal can then decode the received bitstream to obtain the 3D audio data for the recorded program, and then send it to headphones for playback.

[0107] Furthermore, this application can also be applied to other scenarios such as VR scenarios, and this application does not limit it.

[0108] The following are Figure 1a Medium encoder, Figure 1b The first encoder (or second encoder) in the middle, Figure 1c and Figure 1d The encoding process of the 3D audio encoder in 1a; and the decoder in 1a. Figure 1b The first decoder (or the second decoder) in the middle, Figure 1c and Figure 1d The decoding process of the 3D audio decoder in the document will be explained.

[0109] Figure 2 This is a schematic diagram illustrating the encoding process as an example.

[0110] S201, Obtain the data to be encoded.

[0111] In one possible approach, the data to be encoded can be source data, i.e. data collected by the data acquisition module. Source data can include audio data, video data, or images.

[0112] In one possible approach, the data to be encoded can be frequency domain data obtained by frequency domain transformation of the source data, hereinafter referred to as the frequency domain data of the source data.

[0113] In one possible approach, the data to be encoded can be the source data or the frequency domain data of the source data after preprocessing; wherein, preprocessing includes, but is not limited to, framing, windowing, etc., and this application does not limit this.

[0114] It should be noted that when the encoder acquires source data or frequency domain data of source data, the encoder can also perform a preprocessing procedure for the data to be encoded. When the encoder acquires source data or preprocessed frequency domain data of source data, the encoder does not need to perform a preprocessing procedure.

[0115] It should also be noted that the data to be encoded mentioned above refers to the data to be source encoded.

[0116] S202, based on the preset network, perform feature transformation on the data to be encoded to obtain the latent features corresponding to the data to be encoded.

[0117] For example, a pre-set network can refer to a trained network, which in some scenarios can also be called an AI encoding network, and can be used for redundancy removal, decorrelation, etc. For example, a pre-set network can be implemented using a neural network, such as a neural network composed of convolutional layers (or deconvolutional layers) and activation layers interleaved and cascaded.

[0118] For example, a neural network consisting of four convolutional layers and three activation layers can be used, such as convolutional layer-activation layer-convolutional layer-activation layer-convolutional layer-activation layer-convolutional layer. The kernel size of each convolutional layer can be set to 5, the number of output feature channels can be set to K (K is a positive integer), and each convolutional layer undergoes a 2x downsampling. Exemplarily, this application does not limit the number of convolutional layers, the number of channels, the downsampling factor, the number of downsampling iterations, etc., nor does it limit the number of activation layers; it should be understood that this application also does not limit the types and number of network layers constituting the neural network.

[0119] For example, after obtaining the data to be encoded, the data to be encoded can be input into a preset network, which will then perform feature transformation on the data to be encoded to remove redundancy and correlation between the data, thereby making the data to be encoded better compressed.

[0120] For example, after performing feature transformation on the data to be encoded, the latent features corresponding to the data to be encoded can be obtained.

[0121] For example, when the data to be encoded is audio data, the latent features corresponding to the data to be encoded can be a matrix of size 1*H, that is, a vector; where H is a positive integer.

[0122] For example, when the data to be encoded is video data or an image, the latent features corresponding to the data to be encoded can be a matrix of size K1*K2; where K1 and K2 are both integers greater than 1.

[0123] S203, perform wavelet decomposition on the latent features to obtain N wavelet coefficient groups.

[0124] For example, after obtaining the latent features, wavelet decomposition can be performed on the latent features based on wavelet functions, decomposing the latent features into N wavelet coefficient groups, each with a size smaller than the latent features. Here, N is an integer greater than 1, and each wavelet coefficient group may include at least one wavelet coefficient.

[0125] For example, when the data to be encoded is audio data, each wavelet coefficient group is a matrix of size 1*h, which is a vector; where h is a positive integer and h is less than H.

[0126] For example, when the data to be encoded is video data or an image, each wavelet coefficient group can be a matrix of size k1*k2; where k1 and k2 are both integers greater than 1, k1 is less than or equal to K1, and k2 is less than or equal to K2; it should be noted that when k1 is equal to K1, k2 is not equal to K2.

[0127] It should be noted that the sizes of the N wavelet coefficient groups can be the same or different, and this application does not impose any restrictions on this.

[0128] S204 performs vector quantization on each of the N wavelet coefficient groups to obtain the bitstream of the data to be encoded.

[0129] For example, vector quantization can be performed on N wavelet coefficient groups according to one or more pre-trained codebooks to obtain the bitstream of the data to be encoded.

[0130] For example, the size of the codebook can be a*b, where a is the length of the codewords in the codebook (where each codeword is a vector), and b is the number of codewords in the codebook. The length of each codeword in a codebook can be equal to the length of at least one wavelet coefficient group; when the wavelet coefficient group is a k1*k2 matrix, the length of the wavelet coefficient group can refer to the number of rows k1 or the number of columns k2 of the wavelet coefficient group.

[0131] For example, when the wavelet coefficient group is a vector of 1*h, for the i-th wavelet coefficient group (i is an integer between 1 and N) in the N wavelet coefficient groups, the codeword with the smallest quantization error with the i-th wavelet coefficient group (that is, the vector quantization value corresponding to the i-th wavelet coefficient group) can be found from the codebook obtained from the pre-trained training; and the vector quantization value index of the i-th wavelet coefficient group in the codebook can be determined.

[0132] For example, when the wavelet coefficient set is a k1*k2 matrix, for the i-th wavelet coefficient set among the N wavelet coefficient sets, the codeword with the smallest quantization error to each row vector (or column vector) in the i-th wavelet coefficient set can be found from the codebook obtained from the pre-trained training (that is, the vector quantization value corresponding to each row vector (or column vector) in the i-th wavelet coefficient set); and the vector quantization value index of each row vector (or column vector) in the i-th wavelet coefficient set in the codebook can be determined.

[0133] For example, after determining the vector quantization value indices corresponding to the N wavelet coefficient groups, the bitstream of the data to be encoded can be generated based on the vector quantization value indices corresponding to the N wavelet coefficient groups.

[0134] In this way, by decomposing a large matrix into multiple smaller matrices before performing vector quantization, the size of the codebook used for vector quantization can be reduced at the same code rate, thereby reducing the storage space occupied by the codebook in the encoding end.

[0135] For example, assuming the bitstream is 16 * 50 = 800 bps, meaning 50 frames of audio data can be transmitted per second, transmitting one frame requires 16 bits. If the latent feature corresponding to one frame of audio data is a 1 * 16 vector, then the codebook size required for vector quantization is 16 * 2. 16 If the latent features corresponding to a frame of audio data are decomposed using wavelet decomposition to obtain four wavelet coefficient groups, and each wavelet coefficient group is a 1*4 vector, then the codebook size required for vector quantization is 4*2. 4 Therefore, it can be seen that by decomposing a large matrix into multiple smaller matrices before performing vector quantization, the required codebook size is greatly reduced.

[0136] Figure 3This is a schematic diagram illustrating an exemplary decoding process. Figure 3 In the embodiments described, the relationship with Figure 2 The decoding process corresponding to the encoding process in the embodiment.

[0137] S301, obtain the bitstream to be decoded.

[0138] For example, the bitstream to be decoded can be an audio data bitstream, a video data bitstream, or an image bitstream; this application does not limit this.

[0139] It should be noted that the bitstream to be decoded mentioned above refers to the bitstream to be decoded from the source.

[0140] S302 performs vector inverse quantization based on the code stream to obtain N wavelet coefficient groups.

[0141] For example, vector dequantization can be performed on the codestream based on one or more pre-trained codebooks to obtain N groups of wavelet coefficients, where N is an integer greater than 1.

[0142] For example, Figure 3 The codebook used for vector dequantization in the embodiment is the same as... Figure 2 The same codebook is used for vector quantization in the embodiments.

[0143] For example, the bitstream contains vector quantization value indices. Therefore, during vector dequantization, the vector quantization value corresponding to the vector quantization value index can be found in the codebook based on the vector quantization value index in the bitstream, and thus N wavelet coefficient groups can be obtained.

[0144] It should be noted that, Figure 3 The N wavelet coefficient groups in the embodiment and Figure 2 The N wavelet coefficient groups in the embodiments are different. Figure 3 Each wavelet coefficient in the N wavelet coefficient groups in the embodiment is Figure 2 The vector quantization value of each wavelet coefficient in the N wavelet coefficient groups in the embodiment.

[0145] S303 performs wavelet synthesis on N wavelet coefficient groups to obtain hidden features.

[0146] For example, after obtaining N sets of wavelet coefficients, wavelet synthesis can be performed on these N sets of wavelet coefficients to obtain latent features.

[0147] For example, Figure 3 The wavelet synthesis process in the embodiment is Figure 2 The inverse process of wavelet decomposition in the embodiment.

[0148] For example, the encoder can share wavelet decomposition parameters with the decoder in advance, or the wavelet decomposition parameters can be added to the bitstream; in this way, the decoder can perform wavelet synthesis on N wavelet coefficient groups according to the wavelet decomposition parameters to obtain the latent features.

[0149] It should be noted that, Figure 3 The hidden features obtained in the embodiments, and Figure 2 The hidden features obtained in the embodiments differ.

[0150] S304, perform feature transformation on the latent features according to the preset network to obtain reconstructed data.

[0151] For example, Figure 3 The preset network in the embodiments can also be a pre-trained network, which in some scenarios can be called an AI decoding network and can be used for feature synthesis, etc. For example, the preset network can be implemented using a neural network, such as a neural network composed of convolutional layers (deconvolutional layers) and activation layers interleaved and cascaded.

[0152] For example, a neural network consisting of four convolutional layers and three activation layers can be used, such as convolutional layer-activation layer-convolutional layer-activation layer-convolutional layer-activation layer-convolutional layer; wherein the kernel size of each convolutional layer can be set to 5, the number of output feature channels can be set to K (K is a positive integer), and each convolutional layer is downsampled by a factor of 2. Exemplarily, this application does not limit the number of convolutional layers, the number of channels, the downsampling factor, the number of downsampling iterations, etc., nor does it limit the number of activation layers; it should be understood that this application also does not limit the types and number of network layers constituting the neural network.

[0153] It should be noted that, Figure 3 In the embodiments, a preset network is used, and Figure 2 In the embodiments, the preset network can be a network with the same structure but different network parameters; of course, Figure 3 In the embodiments, a preset network is used, and Figure 2 In the embodiments, the preset network can be a network with a different structure, and this application does not limit it.

[0154] For example, latent features can be input into a preset network, which then performs feature synthesis to reconstruct the data and obtain the reconstructed data.

[0155] For example, when the bitstream to be decoded is an audio data bitstream, the reconstructed data is audio data; when the bitstream to be decoded is a video data bitstream, the reconstructed data is video data; and when the bitstream to be decoded is an image bitstream, the reconstructed data is image.

[0156] Thus, compared to performing vector dequantization on the bitstream obtained by vector quantization of a large matrix, the codebook size required for vector dequantization of multiple small matrix vector quantization bitstreams is smaller, thereby reducing the storage space occupied by the codebook at the decoding end.

[0157] For example, during the encoding process, the latent features can be decomposed into N wavelet coefficient groups. This can further reduce the size of the wavelet coefficient groups, thereby reducing the size of the codebook required for vector quantization / vector dequantization, and thus further reducing the storage space occupied by the codebook at the encoding and decoding end.

[0158] Figure 4a This is a schematic diagram of wavelet decomposition as an example. Figure 4a The embodiment shows a schematic diagram of three-level wavelet decomposition; and the second-level and third-level wavelet decompositions only perform wavelet decomposition on a portion of the wavelet coefficient groups obtained from the previous-level wavelet decomposition.

[0159] Reference Figure 4a For example, the first-level wavelet decomposition: wavelet decomposition of the latent features yields two wavelet coefficient sets: cd1 and ca1; where the frequency band corresponding to cd1 is higher than the frequency band corresponding to ca1.

[0160] Second-level wavelet decomposition: Wavelet decomposition of wavelet coefficient group ca1 yields two wavelet coefficient groups cd2 and ca2; among them, the frequency band corresponding to cd2 is higher than the frequency band corresponding to ca2.

[0161] Third-level wavelet decomposition: Wavelet decomposition of wavelet coefficient group ca2 yields two wavelet coefficient groups cd3 and ca3; among them, the frequency band corresponding to cd3 is higher than that corresponding to ca2.

[0162] Thus, by performing a three-level wavelet decomposition on the latent features, we can finally obtain four sets of wavelet coefficients: cd1, cd2, cd3, and ca3.

[0163] Figure 4b This is a schematic diagram of wavelet decomposition as an example. Figure 4b The embodiment shows a schematic diagram of three-level wavelet decomposition; and the second-level wavelet decomposition and the third-level wavelet decomposition perform wavelet decomposition on all wavelet coefficient groups obtained from the previous-level wavelet decomposition.

[0164] Reference Figure 4b For example, the first-level wavelet decomposition: wavelet decomposition of the latent features yields two wavelet coefficient sets: kd11 and ka11; where the frequency band corresponding to kd11 is higher than the frequency band corresponding to ka1.

[0165] Second-level wavelet decomposition: Wavelet decomposition of wavelet coefficient set ka11 yields two wavelet coefficient sets kd22 and ka22; the frequency band corresponding to kd22 is higher than that corresponding to ka22. Similarly, wavelet decomposition of wavelet coefficient set kd11 yields two wavelet coefficient sets kd21 and ka21; the frequency band corresponding to kd21 is higher than that corresponding to ka21. This process can be repeated for kd21, ka21, kd22, and ka22, resulting in the third-level wavelet decomposition.

[0166] Thus, by performing a three-level wavelet decomposition on the latent features, we can finally obtain eight sets of wavelet coefficients: kd31, ka31, kd32, ka32, kd33, ka33, kd34, and ka34.

[0167] It should be understood that, Figure 4a and Figure 4b This is merely an example of the multi-level wavelet decomposition in this application; the application may also perform wavelet decomposition of more or fewer levels on latent features.

[0168] In addition, this application can also use other methods to perform wavelet decomposition, for example, Figure 4a When performing the third-level wavelet decomposition, wavelet decomposition is also performed on cd2; or in Figure 4b In the third-level wavelet decomposition, wavelet decomposition is not performed on kd21 and kd22; this application does not impose any restrictions on this.

[0169] It should be noted that during the decoding process, multi-level wavelet synthesis can be performed on the N wavelet coefficient groups to obtain the latent features; specifically, the multi-level wavelet synthesis can be performed as described above. Figure 4a and Figure 4b The inverse process of the multi-level wavelet decomposition shown in the figure will not be described in detail here.

[0170] In one possible approach, the encoder can pre-share with the decoder the vector quantization indexes corresponding to the N wavelet coefficient groups, as well as their positions and lengths within the bitstream. In this way, after the decoder performs vector dequantization based on the bitstream to obtain the N wavelet coefficient groups, it can determine the wavelet coefficient groups required for each stage of wavelet synthesis based on the pre-shared information from the encoder.

[0171] In one possible approach, the encoder can also add N wavelet coefficient groups to the bitstream, specifying the position and length of their corresponding vector quantization values. In this way, after the decoder performs vector inverse quantization on the bitstream to obtain N wavelet coefficient groups, it determines the wavelet coefficient groups required for each level of wavelet synthesis based on the information parsed from the bitstream.

[0172] For example, X1 wavelet coefficient groups out of N wavelet coefficient groups can be vector-split to obtain U1 (U1 is an integer greater than 1) sub-vectors corresponding to each of the X1 wavelet coefficient groups; then, vector quantization is performed on the other X2 wavelet coefficient groups out of the N wavelet coefficient groups, and vector quantization is performed on the U1 sub-vectors corresponding to each of the X1 wavelet coefficient groups to obtain the bitstream of the data to be encoded. Here, X1 is a positive integer less than or equal to N, X2 is an integer less than N, and X1 + X2 = N.

[0173] This allows the wavelet coefficient set vector to be split into smaller sub-vectors; consequently, it further reduces the size of the codebook required for vector quantization, thereby further reducing the storage space occupied by the codebook at the encoding and decoding end. This application uses X1=N, X2=0 as an example for illustrative purposes.

[0174] Figure 5 This is a schematic diagram illustrating an exemplary encoding process. Figure 5 In the embodiments, the encoding process when X1=N and X2=0 is described.

[0175] S501, obtain the data to be encoded.

[0176] S502, based on the preset network, perform feature transformation on the data to be encoded to obtain the latent features corresponding to the data to be encoded.

[0177] S503 performs wavelet decomposition on the latent features to obtain N sets of wavelet coefficients.

[0178] For example, S501 to S503 can refer to S201 to S203 above, and will not be repeated here.

[0179] S504, perform vector splitting on each of the N wavelet coefficient groups to obtain U1 sub-vectors corresponding to each of the N wavelet coefficient groups.

[0180] For example, for the i-th wavelet coefficient group in N wavelet coefficient groups, when the i-th wavelet coefficient group is a vector of length 1*h, the i-th wavelet coefficient group vector can be split into at least two sub-vectors, where the sum of the lengths of all sub-vectors of the i-th wavelet coefficient group is equal to h. In this way, each of these N wavelet coefficient groups can be vector-split into at least two sub-vectors.

[0181] For example, assuming according to Figure 4aBy performing a two-level wavelet decomposition on the latent features, three wavelet coefficient sets can be obtained: cd1, cd2, and ca2; where cd1 has a length of 128, and cd2 and ca2 both have a length of 64. The cd1 vector can be split into 16 sub-vectors, cd2 into 8 sub-vectors, and ca2 into 8 sub-vectors; thus, 32 sub-vectors can be obtained.

[0182] For example, for the i-th wavelet coefficient group in N wavelet coefficient groups, when the i-th wavelet coefficient group is a vector of length k1*k2, it can first be split into k1 vectors of length k2. Then, for the j-th vector (j is an integer between 1 and k1) among the k1 vectors, it can be split into at least two sub-vectors, where the sum of the lengths of all sub-vectors of the j-th vector is equal to k2. In this way, each of the N wavelet coefficient groups can be split into at least four sub-vectors.

[0183] For example, assuming according to Figure 4a By performing a two-level wavelet decomposition on the latent features, three wavelet coefficient groups can be obtained: cd1, cd2, and ca2; where cd1 has a size of 4*128, and cd2 and ca2 both have a length of 4*64.

[0184] For example, the vector cd1 can be split into four vectors of length 128, and then each of these four vectors can be further split into 16 sub-vectors; thus, cd1 can be split into 4 * 16 = 64 sub-vectors. Similarly, the vector cd2 can be split into four vectors of length 64, and then each of these four vectors can be further split into 8 sub-vectors; thus, cd2 can be split into 4 * 8 = 32 sub-vectors. And the vector ca2 can be split into four vectors of length 64, and then each of these four vectors can be further split into 8 sub-vectors; thus, ca2 can be split into 4 * 8 = 32 sub-vectors. Therefore, a total of 128 sub-vectors can be obtained.

[0185] For example, for the i-th wavelet coefficient group in N wavelet coefficient groups, when the i-th wavelet coefficient group is a vector of length k1*k2, it can first be split into k2 vectors of length k1; then, for the g-th vector (g is an integer between 1 and k2) among the k2 vectors, it can be split into at least two sub-vectors, where the sum of the lengths of all sub-vectors of the g-th vector is equal to k1. In this way, each of the N wavelet coefficient groups can be split into at least four sub-vectors.

[0186] For example, assuming according to Figure 4aBy performing a two-level wavelet decomposition on the latent features, three wavelet coefficient groups can be obtained: cd1, cd2, and ca2; where cd1 has a size of 16*128, and cd2 and ca2 both have a length of 16*64.

[0187] For example, the vector cd1 can be split into 128 vectors of length 16, and then each of these 128 vectors can be further split into 4 sub-vectors; thus, cd1 can be split into 4 * 128 = 512 sub-vectors. Similarly, the vector cd2 can be split into 64 vectors of length 16, and then each of these 64 vectors can be further split into 4 sub-vectors; thus, cd2 can be split into 4 * 64 = 256 sub-vectors. And the vector ca2 can be split into 64 vectors of length 16, and then each of these 64 vectors can be further split into 4 sub-vectors; thus, ca2 can be split into 4 * 64 = 256 sub-vectors. Therefore, a total of 1024 sub-vectors can be obtained.

[0188] It should be noted that when the i-th wavelet coefficient group is a k1*k2 matrix, if the length of a vector after splitting the wavelet coefficient group vector into a vector is less than a preset length, then it is not necessary to split the vector; the vector can be directly determined as a sub-vector of the wavelet coefficient group. The preset length can be set according to requirements, and this application does not impose any restrictions on it.

[0189] It should be noted that the lengths of the U1 subvectors corresponding to the same wavelet coefficient group can be the same or different, and this application does not impose any restrictions on this.

[0190] S504 performs vector quantization on the U1 sub-vectors corresponding to the N wavelet coefficient groups respectively to obtain the code stream of the data to be encoded.

[0191] For example, for the i-th wavelet coefficient group among N wavelet coefficient groups, one or more pre-trained codebooks can be used to perform vector quantization on each sub-vector of the i-th wavelet coefficient group, determining the vector quantization index of each sub-vector in the i-th wavelet coefficient group. Thus, after obtaining the vector quantization indexes of all sub-vectors in the N wavelet coefficient groups, the bitstream of the data to be encoded can be generated based on these indexes.

[0192] Figure 6 This is a schematic diagram illustrating an exemplary decoding process. Figure 6 In the embodiments described, the relationship with Figure 5 The decoding process corresponding to the encoding process.

[0193] S601, obtain the bitstream to be decoded.

[0194] For example, S601 can be described with reference to the above description of S301, and will not be repeated here.

[0195] S602 performs vector inverse quantization based on the code stream to obtain U1 sub-vectors corresponding to N wavelet coefficient groups.

[0196] For example, the encoder splits each wavelet coefficient group vector into U1 sub-vectors, and then performs vector quantization on each sub-vector. The resulting bitstream can include the vector quantization indices of the U1 sub-vectors corresponding to the N wavelet coefficient groups. Subsequently, after acquiring the bitstream and performing vector inverse quantization, the decoder can obtain the U1 sub-vectors corresponding to the N wavelet coefficient groups.

[0197] It should be noted that, Figure 6 The sub-vectors in the embodiments, and Figure 5 The sub-vectors in the embodiments differ; Figure 6 The sub-vector in the embodiment is Figure 5 The vector quantization value of the subvector in the example.

[0198] For example, a subvector can be obtained by looking up the corresponding vector quantization value from a pre-trained codebook based on a vector quantization index in the codestream. Furthermore, in this way, U1 subvectors corresponding to N wavelet coefficient groups can be found from one or more pre-trained codebooks.

[0199] It should be understood that, Figure 6 The codebook used for vector dequantization in the embodiment is the same as... Figure 5 The same codebook is used for vector quantization in the embodiments.

[0200] S603 performs vector merging on the U1 sub-vectors corresponding to the N wavelet coefficient groups to obtain N wavelet coefficient groups.

[0201] In one possible approach, the encoder can pre-share with the decoder the position and length of the vector quantization index of each sub-vector in the bitstream, as well as the wavelet coefficient group to which each sub-vector belongs. In this way, the decoder can use the pre-shared information from the encoder to vector-merge sub-vectors belonging to the same wavelet coefficient group to obtain a single wavelet coefficient group; and thus, N wavelet coefficient groups can be obtained in this manner.

[0202] In one possible approach, the encoder can also add the position and length of the vector quantization index of each subvector in the bitstream, as well as the wavelet coefficient group to which each subvector belongs. In this way, the decoder can use the information parsed from the bitstream to vector-merge subvectors belonging to the same wavelet coefficient group to obtain a single wavelet coefficient group; and thus, N wavelet coefficient groups can be obtained in this manner.

[0203] S604 uses wavelet synthesis based on N wavelet coefficient groups to obtain hidden features.

[0204] S605, based on the preset network, performs feature transformation on the latent features to obtain reconstructed data.

[0205] For example, S604 to S605 can refer to S303 to S304 above, and will not be repeated here.

[0206] It should be noted that the encoding end may split only a portion of the wavelet coefficient group vectors into corresponding U1 sub-vectors, or it may not split any wavelet coefficient group vector into corresponding U1 sub-vectors. The specific split can be determined based on factors such as the importance of each wavelet coefficient group to the data reconstruction quality, the size of the wavelet coefficient group, device performance, and application scenario. This application does not impose any restrictions on this. For example, the number of sub-vectors U1 corresponding to each wavelet coefficient group can be the same or different; this application does not impose any restrictions on this.

[0207] It should be noted that if X1 is less than N and X2 is greater than 0, the decoder performs vector inverse quantization based on the bitstream, which yields U1 sub-vectors corresponding to X1 wavelet coefficient groups and X2 wavelet coefficient groups respectively. Then, the U1 sub-vectors corresponding to the X1 wavelet coefficient groups can be vector-merged to obtain X1 wavelet coefficient groups. In this way, the decoder can obtain N wavelet coefficient groups.

[0208] For example, M codebooks can be pre-trained for N wavelet coefficient groups, where M is a positive integer less than or equal to N. In this way, during the vector quantization of the N wavelet coefficient groups, the target codebooks corresponding to each of the N wavelet coefficient groups can be determined from the pre-trained M codebooks. Then, based on the target codebooks corresponding to the N wavelet coefficient groups, vector quantization is performed on each of the N wavelet coefficient groups to obtain the bitstream of the data to be encoded.

[0209] For example, the size of the M-group codebook can be designed first based on the N wavelet coefficient groups, and a first preset relationship can be established to describe the relationship between the M-group codebook and the N wavelet coefficient groups. Then, based on the first preset relationship, and in conjunction with the AI ​​encoding network and the AI ​​decoding network, the M-group codebook can be trained to obtain the quantized values ​​of each vector in the M-group codebook.

[0210] For example, when M equals N, a codebook can correspond to a set of wavelet coefficients. For the i-th wavelet coefficient set among N wavelet coefficient sets, the length and number of codewords in the m-th codebook (where m is an integer between 1 and M) corresponding to the i-th wavelet coefficient set can be determined based on the size of the i-th wavelet coefficient set and the importance of the i-th wavelet coefficient set to the data reconstruction quality (i.e., the size of the m-th codebook).

[0211] For example, if the i-th wavelet coefficient group does not need to be split into U1 sub-vectors, then the m-th codebook may contain only one codebook. In this case, the length of each codeword in the m-th codebook can be determined based on the size of the i-th wavelet coefficient group; and the number of codewords in the m-th codebook can be determined based on the importance of the i-th wavelet coefficient group to the data reconstruction quality. Next, preset values ​​can be used to assign values ​​to each codeword in the m-th codebook, and a group mapping relationship between the i-th wavelet coefficient group and the m-th codebook can be established. Furthermore, in this way, a group mapping relationship can be established between M codebooks and N wavelet coefficient groups.

[0212] For example, the number of codewords in the codebook corresponding to wavelet coefficient groups that are of high importance to data reconstruction quality can be greater than the number of codewords in the codebook corresponding to wavelet coefficient groups that are of low importance to data reconstruction quality. This can improve the data reconstruction quality. It should be understood that the number of codewords in the codebook corresponding to wavelet coefficient groups that are of high importance to data reconstruction quality can also be equal to the number of codewords in the codebook corresponding to wavelet coefficient groups that are of low importance to data reconstruction quality, and this application does not impose any limitation on this.

[0213] For example, as mentioned above Figure 4a As shown, if N equals 4, then M can also equal 4. In this case, a set of codebooks m1, m2, m3, and m4 can be designed for the wavelet coefficient groups cd1, cd2, cd3, and ca3 respectively; and the group mapping relationship between the four codebooks and the four wavelet coefficient groups is established, as shown in Table 1:

[0214] Table 1

[0215] wavelet coefficient group Codebook Group cd1 m1 cd2 m2 cd3 m3 ca3 m4

[0216] For example, in Table 1, the length of the codeword in codebook m1 corresponds to the size of cd1, the length of the codeword in codebook m2 corresponds to the size of cd2, the length of the codeword in codebook m3 corresponds to the size of cd3, and the length of the codeword in codebook m4 corresponds to the size of ca3.

[0217] For example, in Table 1, if the wavelet coefficient groups cd1, cd2, cd3, and ca3 increase in importance to the data reconstruction quality in that order, then the number of codewords in codebooks m1, m2, m3, and m4 can increase in that order as well. Of course, the number of codewords in codebooks m1, m2, m3, and m4 can also be the same.

[0218] For example, if the i-th wavelet coefficient group needs to be split into v1 (v1 is an integer greater than 1) sub-vectors, then the m-th codebook can include v2 (v2 is a positive integer, v2 is less than or equal to v1) codebooks. When v2 equals v1, the length of each codeword in the corresponding s2 (s2 is an integer between 1 and v2) codebook can be determined based on the length of the s1 (s1 is an integer between 1 and v1) sub-vector in the v1 sub-vectors; and the number of codewords in the s2 codebook can be determined according to requirements. Then, preset values ​​can be used to assign values ​​to each codeword in the s2 codebook, and an element mapping relationship between the s1 sub-vector and the s2 codebook can be established (where each sub-vector can be called an element in its respective wavelet coefficient group, and a codebook can be called an element in its respective set of codebooks).

[0219] When v2 is less than v1, the length of each codeword in the corresponding s2-th codebook can be determined based on the lengths of the y1 sub-vectors among the v1 sub-vectors (these y1 sub-vectors have the same length, and y1 is a positive integer less than or equal to v1); and the number of codewords in the s2-th codebook can be determined according to requirements. Then, preset values ​​can be used to assign values ​​to each codeword in the s2-th codebook, and an element-mapping relationship between the y1 sub-vectors and the s2-th codebook can be established. Furthermore, in this way, an element-mapping relationship can be established between multiple codebooks corresponding to M sets of codebooks and the U1 sub-vectors corresponding to N wavelet coefficient sets.

[0220] It should be noted that the number of codewords in the codebook corresponding to each of the U1 sub-vectors obtained by splitting the same wavelet coefficient group vector can be the same or different; the specific settings can be made according to the requirements, and this application does not impose any restrictions on this.

[0221] It should be noted that when the frequency band corresponding to the r-th wavelet coefficient group (r is a positive integer between 1 and N, and r is not equal to i) is lower than the frequency band corresponding to the i-th wavelet coefficient group, it means that the r-th wavelet coefficient group is more important to the data reconstruction quality than the i-th wavelet coefficient group. In this case, the number of codewords in the codebook corresponding to each of the U1 sub-vectors obtained by splitting the r-th wavelet coefficient group vector can be greater than the number of codewords in the codebook corresponding to each of the U1 sub-vectors obtained by splitting the i-th wavelet coefficient group vector.

[0222] For example, as mentioned above Figure 4a As shown, if N equals 4, then M can also equal 4. In this case, a set of codebooks m1, m2, m3, and m4 can be designed for the wavelet coefficient sets cd1, cd2, cd3, and cd4, respectively. Assume that the wavelet coefficient sets cd1, cd2, cd3, and cd4 are all vectors, with cd1 having a length of 256, cd2 having a length of 128, cd3 having a length of 64, and ca3 having a length of 64. cd1 is split into 32 sub-vectors, cd2 into 16 sub-vectors, cd3 into 8 sub-vectors, and ca3 into 8 sub-vectors. If each sub-vector corresponds to one codebook, the element mapping relationship between the 64 sub-vectors and the 64 codebooks can be shown in Table 2.

[0223] Table 2

[0224]

[0225]

[0226] For example, in Table 2, the length of the codeword in codebook s21_1 corresponds to the size of s11_1, the length of the codeword in codebook s21_2 corresponds to the size of s11_2, ... the length of the codeword in codebook s21_32 corresponds to the size of s11_32; and so on.

[0227] For example, in Table 2, if the importance of wavelet coefficient groups s11_1, s11_2, ..., s11_32 to the data reconstruction quality increases sequentially, then the number of codewords in codebooks s21_1, s21_2, ..., and s21_32 can increase sequentially. Of course, the number of codewords in codebooks s21_1, s21_2, ..., and s21_32 can also be the same; and so on.

[0228] If multiple subvectors correspond to one codebook, and the M codebooks include 56 codebooks, then the element mapping relationship between the 64 subvectors and the 56 codebooks can be shown in Table 3:

[0229] Table 3

[0230]

[0231]

[0232] For example, in Table 3, s11_1 and s11_2 have the same size, and the length of the codeword in codebook s21_1 corresponds to the size of either s11_1 or s11_2. s11_3 and s11_4 have the same size, and the length of the codeword in codebook s21_2 corresponds to the size of either s11_3 or s11_4. The length of the codeword in codebook s21_5 corresponds to the size of s11_3, ..., the length of the codeword in codebook s21_30 corresponds to the size of s11_32; and so on.

[0233] For example, in Table 3, if the importance of wavelet coefficient groups s11_1, s11_2, ..., s11_32 to the data reconstruction quality increases sequentially, then the number of codewords in codebooks s21_1, s21_2, ..., and s21_30 can increase sequentially. Of course, the number of codewords in codebooks s21_1, s21_2, ..., and s21_30 can also be the same; and so on.

[0234] For example, when M is less than N, a set of codebooks can correspond to multiple wavelet coefficient sets. For the y2 wavelet coefficient sets (y2 is a positive integer less than or equal to N) among the N wavelet coefficient sets, the length and number of codewords in the m-th codebook (m is an integer between 1 and M) corresponding to the y2 wavelet coefficient sets can be determined based on the size of the y2 wavelet coefficient sets and the average importance of the y2 wavelet coefficient sets to the data reconstruction quality. Specifically, the group mapping relationship between the m-th codebooks corresponding to the i-th wavelet coefficient set can be established as described above, or the element mapping relationship between the multiple codebooks corresponding to the M codebooks and the U1 sub-vectors corresponding to the N wavelet coefficient sets can be established respectively. These details will not be elaborated here.

[0235] For example, as mentioned above Figure 4a As shown, if N equals 4, then M can equal 3. In this case, wavelet coefficient groups cd1 and cd2 can correspond to a codebook m1, and cd3 and ca3 can correspond to codebooks m2 and m3 respectively; and the group mapping relationship between the 3 codebooks and the 4 wavelet coefficient groups is established, as shown in Table 4:

[0236] Table 4

[0237]

[0238]

[0239] For example, in Table 4, cd1 and cd2 have the same size, the length of the codeword in codebook m1 corresponds to the size of cd1 or cd2, the length of the codeword in codebook m2 corresponds to the size of cd3, and the length of the codeword in codebook m3 corresponds to the size of ca3.

[0240] For example, in Table 4, if the wavelet coefficient groups cd1, cd2, cd3, and ca3 increase in importance to the data reconstruction quality in that order, then the number of codewords in codebooks m1, m2, and m3 can increase in that order as well. Of course, the number of codewords in codebooks m1, m2, and m3 can also be the same.

[0241] For example, the group mapping relationship between M codebooks and N wavelet coefficient groups, or the element mapping relationship between multiple codebooks corresponding to M codebooks and U1 sub-vectors corresponding to the N wavelet coefficient groups respectively, can be called the first preset relationship.

[0242] For example, multiple sets of training data can be collected, which may include audio data, video data, and images. The following uses a single set of training data as an example to illustrate how to train an M-group codebook. One possible approach is to input the training data into an AI encoding network for feature transformation to obtain latent features. Next, wavelet decomposition is performed on the latent features to obtain N wavelet coefficient groups. Then, based on a pre-defined M-group codebook and the group mapping relationship between the M-group codebook and the N wavelet coefficient groups, vector quantization is performed on the N wavelet coefficient groups to obtain vector quantization indexes. Then, based on the vector quantization indexes, the pre-defined M-group codebook, and the group mapping relationship between the M-group codebook and the N wavelet coefficient groups, vector inverse quantization is performed to obtain N wavelet coefficient groups. Next, wavelet synthesis is performed on the N wavelet coefficient groups to obtain latent features. These latent features are then input into an AI decoding network to obtain the reconstructed data output by the AI ​​decoding network. Next, backpropagation can be performed based on the reconstructed data and training data output by the AI ​​decoding network to adjust the network parameters of the AI ​​encoding network, the network parameters of the AI ​​decoding network, and the values ​​of each codeword in the M-group codebook.

[0243] In one possible approach, a set of training data can be input into an AI encoding network for feature transformation to obtain latent features. Next, wavelet decomposition is performed on the latent features to obtain N wavelet coefficient sets. Subsequently, the N wavelet coefficient sets can be split into corresponding U1 sub-vectors. Based on a pre-defined set of M codebooks and the element-mapping relationships between the M codebooks and the U1 sub-vectors corresponding to the N wavelet coefficient sets, vector quantization is performed on each sub-vector in the N wavelet coefficient sets to obtain a vector quantization index. Then, based on the vector quantization index, the pre-defined set of M codebooks and the element-mapping relationships between the M codebooks and the U1 sub-vectors corresponding to the N wavelet coefficient sets, vector dequantization is performed to obtain the U1 sub-vectors corresponding to the N wavelet coefficient sets. Next, the U1 sub-vectors corresponding to the N wavelet coefficient groups are vector-merged to obtain N wavelet coefficient groups. Then, the N wavelet coefficient groups are synthesized using wavelets to obtain latent features. These latent features are input into the AI ​​decoding network to obtain the reconstructed data output by the AI ​​decoding network. Then, backpropagation is performed based on the reconstructed data output by the AI ​​decoding network and the training data to adjust the network parameters of the AI ​​encoding network, the AI ​​decoding network, and the values ​​of each codeword in the M codebooks.

[0244] Following the above method, after training with multiple sets of training data, we can obtain the trained AI encoding network, the trained AI decoding network, and the trained M-set codebook.

[0245] Figure 7 This is a schematic diagram illustrating an exemplary encoding process. Figure 7 In the embodiments described, an encoding process is described where M equals N and the wavelet coefficient group vector is not split into sub-vectors.

[0246] S701, obtain the data to be encoded.

[0247] S702, based on the preset network, perform feature transformation on the data to be encoded to obtain the latent features corresponding to the data to be encoded.

[0248] S703 performs wavelet decomposition on the latent features to obtain N sets of wavelet coefficients.

[0249] For example, S701 to S703 can be referred to the description of S201 to S203, and will not be repeated here.

[0250] S704, according to the first preset relationship, determine the target codebooks corresponding to N wavelet coefficient groups from the preset M codebooks.

[0251] For example, the encoding end can pre-store a first preset relationship. Then, for the i-th wavelet coefficient group in N wavelet coefficient groups, the target codebook corresponding to the i-th wavelet coefficient group in the M codebooks can be determined according to the group mapping relationship between the M codebooks and the N wavelet coefficient groups. In this way, the target codebooks corresponding to the N wavelet coefficient groups can be determined respectively.

[0252] S705 performs vector quantization on each of the N wavelet coefficient groups based on the target codebook corresponding to the N wavelet coefficient groups to obtain the code stream of the data to be encoded.

[0253] For example, for the i-th wavelet coefficient group among N wavelet coefficient groups, the vector quantization value (i.e., a codeword in the target codebook) corresponding to the i-th wavelet coefficient can be found in the target codebook; and the vector quantization value index corresponding to the i-th wavelet coefficient can be determined. Furthermore, after determining the vector quantization value indexes corresponding to the N wavelet coefficient groups, the code stream of the data to be encoded can be generated based on the vector quantization value index corresponding to the N-th wavelet coefficient group.

[0254] For example, different wavelet coefficient sets have varying degrees of importance to data reconstruction quality. When designing codebooks for different wavelet coefficient sets, the precision (i.e., the number of codewords in the codebook) of the codebook corresponding to wavelet coefficient sets that are highly important for data reconstruction quality can be increased, while the precision of the codebook corresponding to wavelet coefficient sets that are less important for data reconstruction quality can be decreased. Thus, by using different codebooks for vector quantization for different wavelet coefficient sets, the proportion of wavelet coefficient sets that are highly important for data reconstruction quality in the bitstream can be increased at the same bitrate, thereby improving the data reconstruction quality.

[0255] Figure 8 This is a schematic diagram illustrating an exemplary decoding process. Figure 8 In the embodiments, the description and Figure 7 The encoding process corresponds to the decoding process in the embodiment.

[0256] S801, obtain the bitstream to be decoded.

[0257] For example, S801 can be described with reference to the above description of S301, and will not be repeated here.

[0258] S802, according to the first preset relationship, determine the target codebooks corresponding to the N wavelet coefficient groups from the M codebooks.

[0259] S803 performs vector inverse quantization based on the target codebook and code stream corresponding to the N wavelet coefficient groups to obtain the N wavelet coefficient groups.

[0260] For example, after the encoder performs vector quantization on N wavelet coefficient groups according to M codebooks, the decoder can determine the target codebooks corresponding to the N wavelet coefficient groups from the preset M codebooks; then, it performs vector inverse quantization based on the target codebooks and the code stream corresponding to the N wavelet coefficient groups to obtain the N wavelet coefficient groups.

[0261] For example, the decoding end can pre-store a first preset relationship. Then, after acquiring the bitstream, for the i-th wavelet coefficient group among the N wavelet coefficient groups, the target codebook corresponding to the i-th wavelet coefficient group can be found from the M codebooks according to the first preset relationship. Next, according to the vector quantization value index corresponding to the i-th wavelet coefficient group in the bitstream, the vector quantization value corresponding to the i-th wavelet coefficient group is found from the target codebook corresponding to the i-th wavelet coefficient group; in this way, the vector quantization values ​​corresponding to the N wavelet coefficient groups can be obtained respectively.

[0262] S804 uses wavelet synthesis based on N wavelet coefficient groups to obtain hidden features;

[0263] S805 performs feature transformation on the latent features based on a preset network to obtain reconstructed data.

[0264] For example, S804 to S805 can be referred to the description of S303 to S304 above, and will not be repeated here.

[0265] In this way, after vector quantization with a high-precision codebook for wavelet coefficient groups that have a high degree of data reconstruction at the encoding end, the decoding end receives a larger proportion of wavelet coefficient groups that are of greater importance to the data reconstruction quality in the received bitstream at the same bit rate; therefore, the decoding end also uses different codebooks for vector inverse quantization for different wavelet coefficient groups, and then after wavelet synthesis and feature transformation, the data reconstruction quality is even higher.

[0266] Figure 9 This is a schematic diagram illustrating an exemplary encoding process. Figure 9 In the embodiments described, the case where M is less than N and the wavelet coefficient group vector is not split into sub-vectors is described.

[0267] S901, obtain the data to be encoded.

[0268] S902, based on the preset network, perform feature transformation on the data to be encoded to obtain the latent features corresponding to the data to be encoded.

[0269] S903 performs wavelet decomposition on the latent features to obtain N wavelet coefficient groups.

[0270] S904, according to the first preset relationship, determine the target codebooks corresponding to N wavelet coefficient groups from the preset M codebooks.

[0271] S905 performs vector quantization on each of the N wavelet coefficient groups based on the target codebook corresponding to the N wavelet coefficient groups to obtain the code stream of the data to be encoded.

[0272] For example, the implementation methods and effects of S901 to S905 can be referred to the description of S701 to S705, and will not be repeated here.

[0273] Figure 10 This is a schematic diagram illustrating an exemplary decoding process. Figure 10 In the embodiments, the description and Figure 9 The encoding process corresponds to the decoding process in the embodiment.

[0274] S1001, Obtain the bitstream to be decoded.

[0275] S1002, according to the first preset relationship, determine the target codebooks corresponding to the N wavelet coefficient groups from the M codebooks.

[0276] S1003 performs vector inverse quantization based on the target codebook and code stream corresponding to the N wavelet coefficient groups to obtain the N wavelet coefficient groups.

[0277] S1004, wavelet synthesis is performed based on N wavelet coefficient groups to obtain hidden features;

[0278] S1005, perform feature transformation on the latent features according to the preset network to obtain reconstructed data.

[0279] For example, the implementation methods and effects of S1001 to S1005 can be referred to the description of S801 to S805 above, and will not be repeated here.

[0280] It should be understood that after splitting the N wavelet coefficient vectors into U1 sub-vectors, during the encoding process, the target codebook corresponding to each sub-vector can be determined based on the element-mapping relationships between the multiple codebooks corresponding to the M codebooks and the U1 sub-vectors corresponding to the N wavelet coefficients. Then, for each sub-vector, vector quantization can be performed using the target codebook corresponding to that sub-vector. Similarly, during the decoding process, the target codebook corresponding to each sub-vector can be determined based on the element-mapping relationships between the multiple codebooks corresponding to the M codebooks and the U1 sub-vectors corresponding to the N wavelet coefficients. Then, the vector quantization value can be found in the corresponding target codebook based on the vector quantization value index of each sub-vector.

[0281] In one possible approach, after designing the size of the M codebooks based on the N wavelet coefficient groups, it is not necessary to establish a first pre-defined relationship to describe the relationship between the M codebooks and the N wavelet coefficient groups. Instead, the AI ​​encoding network and AI decoding network are directly combined to train the M codebooks to obtain the vector quantization values ​​of each value in the M codebooks. The training process can be as follows: In one possible approach, a set of training data can be input into the AI ​​encoding network for feature transformation to obtain latent features. Then, wavelet decomposition is performed on the latent features to obtain N wavelet coefficient groups. Subsequently, for the i-th wavelet coefficient group among the N wavelet coefficient groups, the group with the smallest vector quantization error between the vector quantization value and the i-th wavelet coefficient group in the M codebooks is determined as the target codebook corresponding to the i-th wavelet coefficient group. In this way, the target codebooks corresponding to the N wavelet coefficient groups can be determined. Then, based on the target codebooks corresponding to the N wavelet coefficient groups, vector quantization is performed on the N wavelet coefficient groups to obtain the vector quantization value index. Then, based on the target codebook corresponding to the N wavelet coefficient groups, vector inverse quantization is performed to obtain N wavelet coefficient groups. Next, wavelet synthesis is performed on the N wavelet coefficient groups to obtain latent features; these latent features are input into the AI ​​decoding network to obtain the reconstructed data output by the AI ​​decoding network. Then, backpropagation is performed based on the reconstructed data output by the AI ​​decoding network and the training data to adjust the network parameters of the AI ​​encoding network, the network parameters of the AI ​​decoding network, and the values ​​of each codeword in the M codebooks.

[0282] In one possible approach, a set of training data can be input into an AI encoding network for feature transformation to obtain latent features. Next, wavelet decomposition is performed on the latent features to obtain N wavelet coefficient sets. Subsequently, these N wavelet coefficient sets can be split into corresponding U1 sub-vectors. For each sub-vector, the codebook with the smallest vector quantization error between its vector quantization value and the sub-vector in M ​​codebooks is identified as the target codebook. This allows for the identification of the target codebooks corresponding to the U1 sub-vectors of each of the N wavelet coefficient sets. Then, based on these target codebooks, vector quantization is performed on the U1 sub-vectors corresponding to each of the N wavelet coefficient sets to obtain vector quantization indexes. Finally, vector inverse quantization is performed based on the target codebooks corresponding to the U1 sub-vectors of each of the N wavelet coefficient sets to obtain the U1 sub-vectors corresponding to each of the N wavelet coefficient sets. Next, the U1 sub-vectors corresponding to the N wavelet coefficient groups are vector-merged to obtain N wavelet coefficient groups. Then, the N wavelet coefficient groups are synthesized using wavelets to obtain latent features. These latent features are input into the AI ​​decoding network to obtain the reconstructed data output by the AI ​​decoding network. Then, backpropagation is performed based on the reconstructed data output by the AI ​​decoding network and the training data to adjust the network parameters of the AI ​​encoding network, the AI ​​decoding network, and the values ​​of each codeword in the M codebooks.

[0283] For example, in the encoding process, one method for determining the target codebooks corresponding to N wavelet coefficient groups from a preset M codebooks can be as follows: For the i-th wavelet coefficient group among the N wavelet coefficient groups: Based on the i-th wavelet coefficient group and the M-group vector quantization values ​​corresponding to the i-th wavelet coefficient group, determine the M-group quantization error, where the M-group vector quantization values ​​are obtained by performing vector quantization on the i-th wavelet coefficient group based on the M-group codebooks; determine the codebook in the M-group codebooks that corresponds to the minimum vector quantization error among the M-group vector quantization errors; and determine the codebook corresponding to the minimum vector quantization error as the target codebook corresponding to the i-th wavelet coefficient group.

[0284] For example, one way to determine the M groups of quantization errors based on the i-th wavelet coefficient group and the M groups of vector quantization values ​​corresponding to the i-th wavelet coefficient group can be by calculating the differences between the i-th wavelet coefficient group and the M groups of vector quantization values ​​corresponding to the i-th wavelet coefficient group, resulting in M ​​differences; these M differences can be determined as the M groups of quantization errors. Another way to determine the M groups of quantization errors based on the i-th wavelet coefficient group and the M groups of vector quantization values ​​corresponding to the i-th wavelet coefficient group can be by calculating the mean square error of the i-th wavelet coefficient group and the M groups of vector quantization values ​​corresponding to the i-th wavelet coefficient group, resulting in M ​​mean square errors; these M mean square errors can be determined as the M groups of quantization errors. It should be understood that other methods can also be used to determine the M groups of quantization errors, and this application does not limit this.

[0285] For example, when the codebook corresponding to the minimum vector quantization error is determined as the target codebook for the i-th wavelet coefficient group, the encoder can add the codebook identifier of the target codebook for each wavelet coefficient group to the bitstream. Then, after receiving the bitstream, the decoder determines the target codebooks for each of the N wavelet coefficient groups based on the codebook identifiers for each of the N wavelet coefficient groups in the bitstream; and then performs vector inverse quantization based on the target codebooks for each of the N wavelet coefficient groups and the bitstream to obtain the N wavelet coefficient groups. The codebook identifier is used to uniquely identify a codebook, and can be, for example, a codebook index.

[0286] It should be understood that during the encoding process, if the N wavelet coefficient groups are split into U1 sub-vectors, the target codebook corresponding to each sub-vector can be determined in the same way as described above. This application does not impose any restrictions on this. Furthermore, the encoder can also add the codebook identifier of a codebook corresponding to each sub-vector to the bitstream. Thus, after receiving the bitstream, the decoder determines the target codebook corresponding to each of the N wavelet coefficient groups' U1 sub-vectors based on the codebook identifiers of each sub-vector. Then, it performs vector inverse quantization based on the target codebooks and the bitstream to obtain the U1 sub-vectors corresponding to the N wavelet coefficient groups.

[0287] In many cases, the bitrate is adjusted. For example, during bitstream transmission, when channel conditions deteriorate, the bitrate is reduced to ensure smooth playback of the reconstructed data. Similarly, during scene switching, the bitrate may be adjusted to meet the needs of different scenarios. Therefore, this application, when the bitrate is reduced, can discard some wavelet coefficient groups or some sub-vectors within those wavelet coefficient groups, without performing vector quantization on these wavelet coefficient groups or sub-vectors; when the bitrate is increased, it restores vector quantization of the discarded wavelet coefficient groups or sub-vectors to adapt to different bitrate encoding.

[0288] Figure 11 This is a schematic diagram illustrating an exemplary encoding process. Figure 11 In one embodiment, the encoding end can adapt to different bit rates for encoding.

[0289] S1101, Obtain the data to be encoded.

[0290] S1102, perform feature transformation on the data to be encoded according to the preset network to obtain the latent features corresponding to the data to be encoded.

[0291] S1103, perform wavelet decomposition on the latent features to obtain N wavelet coefficient groups.

[0292] For example, S1101 to S1103 can be referred to the description of S201 to S202 above, and will not be repeated here.

[0293] S1104, based on the target code rate, select G wavelet coefficient groups to be vector quantized from N wavelet coefficient groups.

[0294] For example, after obtaining N wavelet coefficient groups through wavelet decomposition, G wavelet coefficient groups to be vector quantized can be selected from the N wavelet coefficient groups according to the target bit rate, i.e., some wavelet coefficient groups are discarded. Here, G is a positive integer less than N.

[0295] In one possible approach, a second pre-defined relationship can be established in advance based on the amount of transmitted data in each of the N wavelet coefficient groups (i.e., the amount of data corresponding to the vector quantization value index of each wavelet coefficient group), the importance of the N wavelet coefficient groups to the data reconstruction quality, and various code rates, to describe the relationship between the multiple code rates and the N wavelet coefficient groups.

[0296] For example, referring to the above Figure 4aAssume the wavelet coefficient sets include: cd1, cd2, cd3, and ca3; where cd1 has a length of 128 bits, cd2 has a length of 64 bits, cd3 has a length of 32 bits, and ca3 has a length of 32 bits. If the data transmission amount of cd1 (i.e., the data amount of the vector quantization value index corresponding to cd1) is 128 bits, the data transmission amount of cd2 is 64 bits, the data transmission amount of cd3 is 32 bits, and the data transmission amount of ca3 is 32 bits; that is, transmitting one frame of data requires 256 bits. If 50 frames of data need to be transmitted per second, the bit rate is 12800 bps without discarding wavelet coefficient sets. If the bit rate is reduced to 11200 bps, then 32 bits of data can be discarded per frame. Since the frequency band corresponding to ca3 is lower than that corresponding to cd3, ca3 contributes more to the data reconstruction quality than cd3, and therefore cd3 can be discarded. If the bitrate is reduced to 9600bps, 64 bits of data per frame can be discarded. Since the frequency band corresponding to CD2 is lower than that corresponding to CD3 and also lower than that corresponding to CA3, CD2 can be discarded. Based on this, a second preset relationship can be established, as shown in Table 5:

[0297] Table 5

[0298]

[0299]

[0300] For example, in the “Discard” column of Table 5, “1” indicates that the item is not discarded, and “0” indicates that the item is discarded.

[0301] Furthermore, after obtaining N wavelet coefficient groups, G wavelet coefficient groups can be determined based on the target code rate and the second preset relationship. For example, referring to Table 5, when the target code rate is 12800bps, the four wavelet coefficient groups cd1, cd2, cd3, and ca3 are selected, each constituting G wavelet coefficient groups. When the target code rate is 11200bps, cd1, cd2, and ca3 are selected as G wavelet coefficient groups. When the target code rate is 9600bps, cd1, cd3, and ca3 are selected as G wavelet coefficient groups.

[0302] In one possible approach, it is not necessary to establish a second preset relationship. Instead, the numerical feature information corresponding to each of the N wavelet coefficient groups is determined. Then, based on the target bit rate and the numerical feature information corresponding to each of the N wavelet coefficient groups, G wavelet coefficient groups to be vector quantized are selected from the N wavelet coefficient groups.

[0303] The numerical characteristic information includes at least one of the following: frequency band, energy, variance or mean; it should be understood that the numerical characteristic information may also include other information, which is not limited in this application.

[0304] For example, the target amount of data to be discarded can be determined based on the target bit rate. Based on the target amount of data to be discarded and the transmission data amounts corresponding to the N wavelet coefficient groups, multiple candidate wavelet coefficient groups are selected from the N wavelet coefficient groups. If the transmission data amounts of the multiple candidate wavelet coefficient groups are the same and equal to the target amount of data to be discarded, then a target wavelet coefficient group is selected from the multiple candidate wavelet coefficient groups based on numerical feature information. Then, other wavelet coefficient groups besides the target wavelet coefficient group are selected from the N wavelet coefficient groups to obtain G wavelet coefficient groups. If the multiple candidate wavelet coefficient groups include a first type of wavelet coefficient group and a second type of wavelet coefficient group; wherein the first type of wavelet coefficients are wavelet coefficient groups that have been determined to be discarded, and the transmission data amounts of each wavelet coefficient group in the second type of wavelet coefficient group are the same, and the transmission data amounts of any wavelet coefficient group in the first type of wavelet coefficient group and any wavelet coefficient group in the second type of wavelet coefficient group are equal to the target amount of data to be discarded, then a target wavelet coefficient group is selected from the second type of wavelet coefficient group based on numerical feature information. Then, select other wavelet coefficient groups from the N wavelet coefficient groups, excluding the target wavelet coefficient group and the first type of wavelet coefficient group, to obtain G wavelet coefficient groups.

[0305] Among them, data feature information is related to data reconstruction quality. Wavelet coefficient groups with low importance to data reconstruction quality can be discarded, while wavelet coefficient groups with high importance to data reconstruction quality can be selected. In this way, the impact on data reconstruction quality can be reduced while adapting the code rate.

[0306] For example, if the numerical feature information is a frequency band, then the wavelet coefficient group with the highest corresponding frequency band among multiple candidate wavelet coefficient groups is discarded. (The higher the frequency band, the less important it is to the quality of data reconstruction).

[0307] For example, if the numerical feature information is energy, then the wavelet coefficient group with the lowest corresponding energy among multiple candidate wavelet coefficient groups is discarded. (The lower the energy, the less important it is to the quality of data reconstruction).

[0308] For example, if the numerical feature information is the mean, then the wavelet coefficient group with the lowest mean among multiple candidate wavelet coefficient groups is discarded. (The lower the mean, the less important it is to the quality of data reconstruction).

[0309] S1105 performs vector quantization on the G wavelet coefficient groups to obtain the bitstream of the data to be encoded.

[0310] For example, S1105 can refer to the description of vector quantization of N wavelet coefficient groups above, which will not be repeated here.

[0311] It should be understood that after vector splitting each of the N wavelet coefficient groups to obtain U1 sub-vectors corresponding to each of the N wavelet coefficient groups, U2 (U2 is a positive integer) target sub-vectors to be vector quantized can be selected from the U1 sub-vectors corresponding to each of the N wavelet coefficient groups, based on the target code rate. In one possible approach, a third preset relationship can be established in advance, which can be used to describe the relationship between multiple code rates and multiple sub-vectors; then, based on the target code rate and the third preset relationship, U2 target sub-vectors are selected from the U1 sub-vectors corresponding to each of the N wavelet coefficient groups. In another possible approach, numerical feature information corresponding to each of the N wavelet coefficient groups can be determined, including at least one of the following: frequency band, energy, variance, or mean; based on the target code rate and the numerical feature information corresponding to each of the N wavelet coefficient groups, U2 target sub-vectors are selected from the U1 sub-vectors corresponding to each of the N wavelet coefficient groups. The method for selecting the target sub-vectors is similar to the method for selecting G wavelet coefficient groups described above, and will not be repeated here.

[0312] For example, the encoder selects G wavelet coefficient groups to be vector quantized from N wavelet coefficient groups according to the target bit rate; then, it performs vector quantization on the G wavelet coefficient groups to obtain the bitstream of the data to be encoded, and writes the wavelet coefficient group identifiers of the G wavelet coefficient groups into the bitstream; in this way, after the decoder performs vector inverse quantization to obtain G wavelet coefficient groups, it can perform wavelet synthesis on the G wavelet coefficient groups according to the wavelet coefficient group identifiers to obtain the latent features.

[0313] For example, the encoder selects U2 target sub-vectors to be vector-quantized from U1 sub-vectors corresponding to N wavelet coefficient groups, based on the target bit rate. Then, it performs vector quantization on the U2 target sub-vectors to obtain the bitstream of the data to be encoded. The sub-vector identifiers of the target sub-vectors are then written into the bitstream. In this way, after the decoder performs vector inverse quantization to obtain U2 target sub-vectors, it can perform wavelet synthesis on the U2 target sub-vectors based on their sub-vector identifiers to obtain latent features. The sub-vector identifier can be used to uniquely identify a sub-vector, such as a sub-vector index.

[0314] In this way, multiple preset networks can be trained without the need for training, and multi-rate transmission can be achieved by discarding some sub-vectors or wavelet coefficient groups, which increases the versatility of the coding method in this application.

[0315] Figure 12 This is an exemplary decoding diagram. Figure 12 In one embodiment, a comparison with Figure 11 The encoding process corresponds to the decoding process.

[0316] S1201, Obtain the bitstream to be decoded.

[0317] S1202 performs vector inverse quantization based on the code stream to obtain G wavelet coefficient groups, where G is a positive integer less than N.

[0318] For example, S1201 to S1202 can be referred to the description of S301 to S302 above, and will not be repeated here.

[0319] In this process, the encoder selects G wavelet coefficient groups from N wavelet coefficient groups for vector quantization, resulting in a bitstream containing only G wavelet coefficient groups. Then, the decoder performs vector inverse quantization based on the bitstream to obtain G wavelet coefficient groups.

[0320] S1203, based on the wavelet coefficient group identifier, perform wavelet synthesis on the vector quantized values ​​of the G wavelet coefficient groups to obtain the hidden features.

[0321] For example, the decoder can determine which of the N wavelet coefficient groups sent by the encoder can be determined based on the wavelet coefficient group identifier parsed from the bitstream; then, during each wavelet synthesis stage, the corresponding wavelet coefficient group to be used can be determined based on the wavelet coefficient group identifier, thereby obtaining the latent features.

[0322] S1204, perform feature transformation on the latent features according to the preset network to obtain reconstructed data.

[0323] For example, S1204 can be described with reference to the above description of S304, and will not be repeated here.

[0324] In this way, multi-bitrate transmission can be achieved without training multiple preset networks, increasing the versatility of the decoding method in this application.

[0325] It should be noted that if X1 is less than N and X2 is greater than 0, then when U2 target sub-vectors correspond to X4 (X4 can be less than or equal to X1) wavelet coefficient groups, the encoder can also select X3 wavelet coefficient groups to be vector quantized from the X2 wavelet coefficient groups according to the target bit rate, where X3 is a positive integer less than or equal to X2. In this way, during vector quantization, the encoder can perform vector quantization on the X3 wavelet coefficient groups and on the U2 target sub-vectors to obtain the bitstream of the data to be encoded. Correspondingly, the decoder performs vector inverse quantization based on the bitstream to obtain the U2 target sub-vectors and X3 wavelet coefficient groups corresponding to the X4 wavelet coefficient groups respectively; then, the U2 target sub-vectors corresponding to the X4 wavelet coefficient groups can be vector-merged to obtain the X4 wavelet coefficient groups.

[0326] In one example, Figure 13The schematic block diagram of an embodiment of the present application shows a device 1300, which may include a processor 1301 and a transceiver / transceiver pin 1302, and optionally, a memory 1303.

[0327] The various components of device 1300 are coupled together via bus 1304, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 1304 in the figure.

[0328] Optionally, the memory 1303 can be used to store instructions from the foregoing method embodiments. The processor 1301 can be used to execute the instructions in the memory 1303, control the receive pin to receive signals, and control the transmit pin to transmit signals.

[0329] Device 1300 may be an electronic device or a chip of an electronic device in the above method embodiments.

[0330] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0331] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the encoding / decoding method in the above embodiment.

[0332] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the encoding and decoding method in the above embodiment.

[0333] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the encoding and decoding methods in the above-described method embodiments.

[0334] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0335] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0336] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0337] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0338] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0339] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0340] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0341] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0342] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0343] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0344] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An encoding method, characterized in that, include: Obtain the data to be encoded; The data to be encoded is subjected to feature transformation based on a preset network to obtain the latent features corresponding to the data to be encoded; The latent features are decomposed using wavelet decomposition to obtain N wavelet coefficient groups, where N is an integer greater than 1; Vector quantization is performed on each of the N wavelet coefficient groups to obtain the bitstream of the data to be encoded.

2. The method according to claim 1, characterized in that, The process of performing wavelet decomposition on the latent features to obtain N sets of wavelet coefficients includes: The latent features are subjected to multi-level wavelet decomposition to obtain the N wavelet coefficient groups.

3. The method according to claim 1 or 2, characterized in that, The step of performing vector quantization on the N wavelet coefficient groups to obtain the bitstream of the data to be encoded includes: From the M trained codebooks, determine the target codebooks corresponding to the N wavelet coefficient groups respectively; where M is a positive integer less than or equal to N; Based on the target codebooks corresponding to the N wavelet coefficient groups, vector quantization is performed on the N wavelet coefficient groups respectively to obtain the code stream of the data to be encoded.

4. The method according to claim 3, characterized in that, The step of determining the target codebooks corresponding to the N wavelet coefficient groups from the trained M codebooks includes: Based on the first preset relationship, the target codebooks corresponding to the N wavelet coefficient groups are determined from the M codebooks respectively; The first preset relationship is used to describe the relationship between the M codebooks and the N wavelet coefficient groups.

5. The method according to claim 3, characterized in that, The step of determining the target codebooks corresponding to the N wavelet coefficient groups from the trained M codebooks includes: For the i-th wavelet coefficient group among the N wavelet coefficient groups, where i is an integer between 1 and N: Based on the i-th wavelet coefficient group and the M-group vector quantization values ​​corresponding to the i-th wavelet coefficient group, the M-group vector quantization error is determined, wherein the M-group vector quantization values ​​are obtained by performing vector quantization on the i-th wavelet coefficient group based on the M-group codebooks respectively; Determine the codebook in the M codebooks that corresponds to the smallest vector quantization error among the M codebooks; The codebook corresponding to the minimum vector quantization error is determined as the target codebook corresponding to the i-th wavelet coefficient group.

6. The method according to any one of claims 3 to 5, characterized in that, The precision of some codebooks in the M-group codebooks differs.

7. The method according to claim 1 or 2, characterized in that, The method further includes: The X1 wavelet coefficient groups in the N wavelet coefficient groups are vector split to obtain U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively, where X1 is a positive integer less than or equal to N and U1 is an integer greater than 1; The step of performing vector quantization on the N wavelet coefficient groups to obtain the bitstream of the data to be encoded includes: Vector quantization is performed on the additional X2 wavelet coefficient groups in the N wavelet coefficient groups, and vector quantization is performed on the U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively, so as to obtain the code stream of the data to be encoded; X2 is an integer less than N, X1+X2=N.

8. The method according to claim 7, characterized in that, The step of performing vector quantization on the additional X2 wavelet coefficient groups from the N wavelet coefficient groups, and performing vector quantization on the U1 sub-vectors corresponding to each of the X1 wavelet coefficient groups to obtain the bitstream of the data to be encoded, includes: Based on the target bit rate, select U2 target sub-vectors to be vector quantized from the U1 sub-vectors corresponding to the X1 wavelet coefficient groups respectively, and select X3 wavelet coefficient groups to be vector quantized from the X2 wavelet coefficient groups, where X3 is a positive integer less than or equal to X2, and U2 is a positive integer; Vector quantization is performed on the X3 wavelet coefficient groups and the U2 target sub-vectors to obtain the bitstream of the data to be encoded.

9. The method according to claim 1 or 2, characterized in that, The step of performing vector quantization on the N wavelet coefficient groups to obtain the bitstream of the data to be encoded includes: Based on the target bit rate, select G wavelet coefficient groups to be vector quantized from the N wavelet coefficient groups, where G is a positive integer less than N; Vector quantization is performed on the G wavelet coefficient groups to obtain the code stream of the data to be encoded.

10. The method according to claim 9, characterized in that, The step of selecting G wavelet coefficient groups to be vector quantized from the N wavelet coefficient groups based on the target bit rate includes: Based on the target bit rate and the second preset relationship, the G wavelet coefficient groups are determined; wherein, the second preset relationship is used to describe the relationship between multiple bit rates and the N wavelet coefficient groups.

11. The method according to claim 9, characterized in that, The step of selecting G wavelet coefficient groups to be vector quantized from the N wavelet coefficient groups based on the target code rate includes: Determine the numerical feature information corresponding to the N wavelet coefficient groups respectively, wherein the numerical feature information includes at least one of the following: frequency band, energy, variance or mean; Based on the target code rate and the numerical feature information corresponding to the N wavelet coefficient groups, G wavelet coefficient groups to be vector quantized are selected from the N wavelet coefficient groups.

12. The method according to any one of claims 3 to 6, characterized in that, The bitstream includes: a codebook identifier.

13. The method according to any one of claims 9 to 11, characterized in that, The code stream includes: wavelet coefficient group identifiers.

14. A decoding method, characterized in that, include: Obtain the bitstream to be decoded; Vector inverse quantization is performed on the bitstream to obtain N wavelet coefficient groups, where N is an integer greater than 1; Wavelet synthesis is performed on the N wavelet coefficient groups to obtain the hidden features; The latent features are transformed according to a preset network to obtain reconstructed data.

15. The method according to claim 14, characterized in that, The step of performing wavelet synthesis on the N wavelet coefficient groups to obtain latent features includes: The N wavelet coefficient groups are subjected to multi-level wavelet synthesis to obtain the hidden features.

16. The method according to claim 14 or 15, characterized in that, The vector inverse quantization based on the bitstream to obtain N wavelet coefficient groups includes: From the M trained codebooks, determine the target codebooks corresponding to the N wavelet coefficient groups respectively; where M is a positive integer less than or equal to N; Vector inverse quantization is performed based on the target codebook and the code stream corresponding to the N wavelet coefficient groups to obtain the N wavelet coefficient groups.

17. The method according to claim 16, characterized in that, The step of determining the target codebooks corresponding to the N wavelet coefficient groups from the trained M codebooks includes: Based on the first preset relationship, the target codebooks corresponding to the N wavelet coefficient groups are determined from the M codebooks respectively; The first preset relationship includes the relationship between the M codebooks and the N wavelet coefficient groups.

18. The method according to claim 16, characterized in that, The code stream includes a codebook identifier; The step of determining the target codebooks corresponding to the N wavelet coefficient groups from the trained M codebooks includes: Based on the codebook identifier in the code stream, determine the target codebook corresponding to each of the N wavelet coefficient groups.

19. The method according to any one of claims 16 to 18, characterized in that, The precision of some codebooks in the M-group codebooks differs.

20. The method according to claim 14 or 15, characterized in that, The vector inverse quantization based on the bitstream to obtain N wavelet coefficient groups includes: Vector inverse quantization is performed based on the bitstream to obtain U1 sub-vectors and X2 wavelet coefficient groups corresponding to X1 wavelet coefficient groups respectively, where X1 is a positive integer less than or equal to N, X2 is an integer less than N, X1+X2=N, and U1 is an integer greater than 1. Vector merging is performed on the U1 sub-vectors corresponding to the X1 wavelet coefficient groups to obtain the X1 wavelet coefficient groups.

21. The method according to claim 14, characterized in that, The code stream includes wavelet coefficient group identifiers; The vector inverse quantization based on the bitstream to obtain N wavelet coefficient groups includes: Vector inverse quantization is performed based on the bitstream to obtain G wavelet coefficient groups, where G is a positive integer less than N; The step of performing wavelet synthesis on the N wavelet coefficient groups to obtain latent features includes: Based on the wavelet coefficient group identifier, wavelet synthesis is performed on the G wavelet coefficient groups to obtain the hidden features.

22. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the encoding method according to any one of claims 1 to 13.

23. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the decoding method according to any one of claims 14 to 21.

24. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of the electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the encoding method according to any one of claims 1 to 13.

25. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the decoding method according to any one of claims 14 to 21.

26. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 1 to 21.

27. A computer program product, characterized in that, The computer program product includes a software program that, when executed by a computer or processor, causes the steps of the method described in any one of claims 1 to 21 to be performed.

Images