Coding method and electronic device

Abstract

Embodiments of the present application provide a coding method and an electronic device. The coding method comprises: first, obtaining a to-be-coded frame, the to-be-coded frame being a P frame; then, determining a network parameter set corresponding to the to-be-coded frame from M preset network parameter sets, wherein the M preset network parameter sets correspond to different compression performance information respectively, and M is an integer greater than 1; and then, coding the to-be-coded frame according to the network parameter set corresponding to the to-be-coded frame by using a coding network to obtain a code stream of the to-be-coded frame. In this way, different network parameter sets can be used to code adjacent multiple frames, so that the compression performance of adjacent multiple frames is high and low alternately, a compression performance confrontation is constructed in time sequence, and cumulative error is weakened in structure, so that the compression performance can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to an encoding / decoding method and an electronic device. Background Technology

[0002] With the development of neural networks in the field of image compression and their performance surpassing traditional compression methods, deep learning-based video compression methods are gradually being widely used.

[0003] In deep learning-based video compression and decompression, the reconstructed image of the previous frame participates in the reconstruction of the next frame. Therefore, the quality of the reconstructed image of the previous frame affects the quality of the reconstructed image of the next frame. When the quality of a certain frame deteriorates, cumulative temporal errors will form over time, creating a chain reaction that causes the compression performance of all subsequent frames to decrease. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides an encoding / decoding method and an electronic device. This method can reduce accumulated errors and improve compression performance.

[0005] In a first aspect, embodiments of this application provide an encoding method, which includes: first, acquiring a frame to be encoded, wherein the frame to be encoded is a P-frame (Predictive-frame); next, determining a set of network parameters corresponding to the frame to be encoded from M preset sets of network parameters, wherein the M preset sets of network parameters correspond to different compression performance information, and M is an integer greater than 1; then, encoding a network to encode the frame to be encoded according to the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded. In this way, during the encoding process of the encoding network, different network parameters can be used to encode multiple adjacent frames, so that the compression performance of adjacent frames alternates between high and low, constructing compression performance adversarial in time sequence, thereby structurally reducing the cumulative error and thus improving compression performance.

[0006] For example, the M preset network parameter sets are obtained through M different training methods.

[0007] For example, compression performance information may include image quality and / or compression ratio.

[0008] According to the first aspect, M equals 2, and the M preset network parameter sets include a first preset network parameter set and a second preset network parameter set. From the M preset network parameter sets, the network parameter set corresponding to the frame to be encoded is determined, including: when the frame identifier of the frame to be encoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded; when the frame identifier of the frame to be encoded is even, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded. In this way, it is possible to encode two adjacent frames using different network parameter sets.

[0009] It should be understood that when the frame identifier of the frame to be encoded is odd, the second preset network parameter set can also be determined as the network parameter set corresponding to the frame to be encoded; when the frame identifier of the frame to be encoded is even, the first preset network parameter set can also be determined as the network parameter set corresponding to the frame to be encoded. That is to say, this application does not limit whether the encoding network uses the first preset network parameter set to encode the frame to be encoded with an odd frame identifier, or uses the second preset network parameter set to encode the frame to be encoded with an odd frame identifier.

[0010] For example, the frame identifier of the frame to be encoded can be a POC (Picture Order Count) value or a frame number of the frame to be encoded. This application does not limit this.

[0011] According to the first aspect, or any implementation of the first aspect above, the compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set; or, the compression performance information corresponding to the second preset network parameter set is higher than the compression performance information corresponding to the first preset network parameter set.

[0012] For example, when the compression performance information is image quality, the image quality corresponding to the first preset network parameter set is higher than the image quality corresponding to the second preset network parameter set; or, the image quality corresponding to the second preset network parameter set is higher than the image quality corresponding to the first preset network parameter set.

[0013] For example, when the compression performance information is the compression ratio, the compression ratio corresponding to the first preset network parameter set is higher than the compression ratio corresponding to the second preset network parameter set; or, the compression ratio corresponding to the second preset network parameter set is higher than the compression ratio corresponding to the first preset network parameter set.

[0014] For example, when the compression performance information includes image quality and compression ratio, the image quality corresponding to the first preset network parameter set is higher than the image quality corresponding to the second preset network parameter set, and the compression ratio corresponding to the first preset network parameter set is higher than the compression ratio corresponding to the second preset network parameter set. Alternatively, the image quality corresponding to the second preset network parameter set is higher than the image quality corresponding to the first preset network parameter set, and the compression ratio corresponding to the second preset network parameter set is higher than the compression ratio corresponding to the first preset network parameter set.

[0015] According to the first aspect, or any implementation of the first aspect above, the network parameter set corresponding to the frame to be encoded is determined from M preset network parameter sets, including: when the network parameter sets corresponding to the previous N frames of the frame to be encoded are the same, according to preset rules, one preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be encoded is selected as the network parameter set corresponding to the frame to be encoded; otherwise, the network parameter set corresponding to the previous frame of the frame to be encoded is determined as the network parameter set corresponding to the frame to be encoded, where N is an integer greater than 1. In this way, the same network parameter set can be used to encode each consecutive N frames.

[0016] For example, preset rules can be set in advance, and the specific rules can be set according to the requirements. This application does not limit this. For example, k(i) = k(i-1) + 1, where k(i) is the index of the network parameter set corresponding to the frame to be encoded, k(i-1) is the index of the network parameter set corresponding to the previous frame of the frame to be encoded, and the numbers of the M preset network parameter sets can be from 1 to M.

[0017] According to the first aspect, or any implementation of the first aspect above, determining the network parameter set corresponding to the frame to be encoded from the M preset network parameter sets includes: determining the compression performance information corresponding to each of the M preset network parameter sets; and determining the network parameter set corresponding to the frame to be encoded from the M preset network parameter sets based on the compression performance information corresponding to each of the M preset network parameter sets.

[0018] According to the first aspect, or any implementation of the first aspect above, based on the compression performance information corresponding to the M preset network parameter sets, the network parameter set corresponding to the frame to be encoded is determined from the M preset network parameter sets. This includes: when the compression performance information corresponding to the previous K frames of the frame to be encoded is higher than a threshold, a preset network parameter set whose compression performance information is lower than the threshold from the M preset network parameter sets is used as the network parameter set corresponding to the frame to be encoded; if the compression performance information corresponding to the previous K frames of the frame to be encoded is lower than the threshold, a preset network parameter set whose compression performance information is higher than the threshold from the M preset network parameter sets is used as the network parameter set corresponding to the frame to be encoded, where K is a positive integer; otherwise, if the compression performance information corresponding to the previous frame of the frame to be encoded is lower than the threshold, a preset network parameter set whose compression performance information is lower than the threshold from the M preset network parameter sets is used as the network parameter set corresponding to the frame to be encoded; if the compression performance information corresponding to the previous frame of the frame to be encoded is higher than the threshold, a preset network parameter set whose compression performance information is higher than the threshold from the M preset network parameter sets is used as the network parameter set corresponding to the frame to be encoded. In this way, each consecutive K frames can be encoded using a different network parameter set.

[0019] For example, K can be greater than N, less than N, or equal to N; this application does not impose any restrictions on this.

[0020] According to the first aspect, or any implementation of the first aspect above, a preset network parameter set includes multiple preset network parameter groups; the encoding network includes multiple sub-networks, and the multiple sub-networks correspond to the multiple preset network parameter groups respectively; and the preset network parameter groups of any two preset network parameter sets in the M preset network parameter sets are different.

[0021] According to the first aspect, or any implementation of the first aspect above, the bitstream of the frame to be encoded includes a set identifier of the network parameter set corresponding to the frame to be encoded. In this way, the decoding end can parse the set identifier from the bitstream, thereby determining the set of network parameters used for decoding the bitstream.

[0022] Secondly, embodiments of this application provide a decoding method, which includes: first, acquiring the bitstream of a frame to be decoded, wherein the frame to be decoded is a P-frame; next, determining the network parameter set corresponding to the frame to be decoded from M preset network parameter sets, wherein the M preset network parameter sets correspond to different compression performance information, and M is an integer greater than 1; then, the decoding network decodes the bitstream of the frame to be decoded according to the network parameter set corresponding to the frame to be decoded, so as to obtain the reconstructed image of the frame to be decoded. In this way, the decoding end can correspond to the encoding end and use different network parameter sets to decode the bitstream.

[0023] According to the second aspect, M equals 2, and the M preset network parameter sets include a first preset network parameter set and a second preset network parameter set. From the M preset network parameter sets, the network parameter set corresponding to the frame to be decoded is determined, including: when the frame identifier of the frame to be decoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded; when the frame identifier of the frame to be decoded is even, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded. In this way, it is possible to decode two adjacent frames using different network parameter sets.

[0024] According to the second aspect, or any implementation of the second aspect above, the compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set; or, the compression performance information corresponding to the second preset network parameter set is higher than the compression performance information corresponding to the first preset network parameter set.

[0025] According to the second aspect, or any implementation of the second aspect above, the network parameter set corresponding to the frame to be decoded is determined from the M preset network parameter sets, including: parsing the set identifier from the bitstream of the frame to be decoded; and determining the preset network parameter set that matches the set identifier from the M preset network parameter sets as the network parameter set corresponding to the frame to be decoded.

[0026] According to the second aspect, or any implementation of the second aspect above, the network parameter set corresponding to the frame to be decoded is determined from the M preset network parameter sets, including: when the network parameter sets corresponding to the previous N frames of the frame to be decoded are the same, according to preset rules, one preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be decoded is selected as the network parameter set corresponding to the frame to be decoded; otherwise, the network parameter set corresponding to the previous frame of the frame to be decoded is determined as the network parameter set corresponding to the frame to be decoded, where N is an integer greater than 1.

[0027] Thirdly, this application provides a training method, which includes: first, acquiring training images; then, acquiring a target network, the target network having M pairs of initial network parameter sets, where M is an integer greater than 1; next, updating the M pairs of initial network parameter sets of the target network based on M methods for determining the loss value and the training images, to obtain M pairs of preset network parameter sets; wherein, the target network includes an encoding network and a decoding network, and each pair of preset network parameter sets includes: a preset network parameter set corresponding to the encoding network and a preset network parameter set corresponding to the decoding network. Thus, through joint training of the encoding network and the decoding network, M preset network parameter sets for the encoding network and M preset network parameter sets for the decoding network are obtained.

[0028] For example, in the target network, the decoding network is connected after the encoding network; that is, the output of the encoding network is connected to the input of the decoding network. Here, the input of the encoding network is the input of the target network, and the output of the decoding network is the output of the target network.

[0029] According to the third aspect, based on M methods for determining the loss value and the training images, the M pairs of initial network parameter sets of the target network are updated to obtain M pairs of preset network parameter sets. This includes: for the i-th pair of initial network parameter sets, where i ranges from 1 to M: the i-th pair of initial network parameter sets is loaded into the target network, and the training images are input into the target network to obtain the reconstructed image output by the target network; according to the i-th method for determining the loss value, the loss value corresponding to the i-th pair of initial network parameter sets is determined based on the reconstructed image and the training image; and the i-th pair of initial network parameter sets is updated based on the loss value corresponding to the i-th pair of initial network parameter sets to obtain the i-th pair of preset network parameter sets. Thus, using one method for determining the loss value and the training images, a pair of prediction network parameter sets is trained.

[0030] According to the third aspect, or any implementation of the third aspect above, the target network also has an initial shared network parameter set. Based on M methods for determining the loss value and training images, the M pairs of initial network parameter sets of the target network are updated respectively to obtain M pairs of preset network parameter sets. This further includes: loading the initial shared network parameter set when loading the i-th pair of initial network parameter sets into the target network; updating the i-th pair of initial network parameter sets according to the loss value corresponding to the i-th pair of initial network parameter sets, and performing the i-th update on the initial shared network parameter set according to the loss value corresponding to the i-th pair of initial network parameter sets; adding the preset shared network parameter sets obtained from the M updates to the initial shared network parameter set to the M pairs of preset network parameter sets respectively. In this way, by alternately training the initial shared network parameter set using M methods for determining the loss value to obtain the preset shared network parameter sets, the training speed can be improved.

[0031] According to the third aspect, or any of the above implementations of the third aspect, the M determination methods correspond to M loss functions, and / or; the M determination methods correspond to M sets of computational data, and the M sets of computational data are used to calculate the M sets of loss values.

[0032] Fourthly, embodiments of this application provide an encoding system, which includes: a selector and an encoding network, wherein:

[0033] The selector is used to determine the set of network parameters corresponding to the frame to be encoded from M preset network parameter sets. The frame to be encoded is a P-frame, and the M preset network parameter sets correspond to different compression performance information. M is an integer greater than 1.

[0034] An encoding network is used to encode a frame based on the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded.

[0035] According to the fourth aspect, the selector can be used to execute the encoding method corresponding to any of the implementations of the first aspect mentioned above.

[0036] The technical effects of the fourth aspect and any implementation thereof can be found in the first aspect and any implementation thereof, and will not be repeated here.

[0037] Fifthly, embodiments of this application provide a decoding system, which includes: a selector and a decoding network, wherein:

[0038] The selector is used to determine the set of network parameters corresponding to the frame to be decoded from M preset network parameter sets. The frame to be decoded is a P-frame, and the M preset network parameter sets correspond to different compression performance information, where M is an integer greater than 1.

[0039] The decoding network is used to decode the bitstream of the frame to be decoded based on the set of network parameters corresponding to the frame to be decoded, so as to obtain the reconstructed image of the frame to be decoded.

[0040] According to the fifth aspect, the selector can be used to execute the decoding method corresponding to any of the implementations of the second aspect above.

[0041] The technical effects of the fifth aspect and any implementation thereof can be found in the technical effects of the second aspect and any implementation thereof, which will not be repeated here.

[0042] In a sixth aspect, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, when executed by the processor, causing the electronic device to perform the encoding method in the first aspect or any possible implementation of the first aspect.

[0043] The sixth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the sixth aspect and any implementation thereof are similar to those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0044] In a seventh aspect, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the electronic device to perform the decoding method in the second aspect or any possible implementation thereof.

[0045] The seventh aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the seventh aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0046] Eighthly, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the electronic device to perform the training method in the third aspect or any possible implementation of the third aspect.

[0047] The eighth aspect and any implementation thereof correspond to the third aspect and any implementation thereof, respectively. The technical effects of the eighth aspect and any implementation thereof can be found in the technical effects of the third aspect and any implementation thereof, as described above, and will not be repeated here.

[0048] Ninthly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to execute the encoding method in the first aspect or any possible implementation of the first aspect.

[0049] The ninth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the ninth aspect and any implementation thereof are similar to those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0050] In a tenth aspect, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the decoding method in the second aspect or any possible implementation of the second aspect.

[0051] The tenth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the tenth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0052] Eleventhly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to execute the training method in the third aspect or any possible implementation of the third aspect.

[0053] The eleventh aspect and any implementation thereof correspond to the third aspect and any implementation thereof, respectively. The technical effects corresponding to the eleventh aspect and any implementation thereof can be found in the technical effects corresponding to the third aspect and any implementation thereof, and will not be repeated here.

[0054] In a twelfth aspect, embodiments of this application provide a computer storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to execute the encoding method in the first aspect or any possible implementation thereof.

[0055] The twelfth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the twelfth aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0056] In a thirteenth aspect, embodiments of this application provide a computer storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to execute the decoding method in the second aspect or any possible implementation thereof.

[0057] The thirteenth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the thirteenth aspect and any implementation thereof are as described above for the technical effects corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0058] In a fourteenth aspect, embodiments of this application provide a computer storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to execute the training method in the third aspect or any possible implementation thereof.

[0059] The fourteenth aspect and any implementation thereof correspond to the third aspect and any implementation thereof, respectively. The technical effects of the fourteenth aspect and any implementation thereof are similar to those of the third aspect and any implementation thereof, and will not be repeated here.

[0060] In a fifteenth aspect, embodiments of this application provide a computer program product comprising a software program that, when executed by a computer or processor, causes the steps of the method in the first aspect or any possible implementation thereof to be performed.

[0061] The fifteenth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the fifteenth aspect and any implementation thereof are similar to those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0062] In a sixteenth aspect, embodiments of this application provide a computer program product comprising a software program that, when executed by a computer or processor, causes the steps of the method in the second aspect or any possible implementation thereof to be performed.

[0063] The sixteenth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the sixteenth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0064] In a seventeenth aspect, embodiments of this application provide a computer program product comprising a software program that, when executed by a computer or processor, causes the steps of the method in the third aspect or any possible implementation thereof to be performed.

[0065] The seventeenth aspect and any implementation thereof correspond to the third aspect and any implementation thereof, respectively. The technical effects corresponding to the seventeenth aspect and any implementation thereof are similar to those corresponding to the third aspect and any implementation thereof, and will not be repeated here. Attached Figure Description

[0066] Figure 1 A schematic diagram illustrating an application scenario is provided as an example.

[0067] Figure 2 This is a schematic diagram illustrating the training process as an example.

[0068] Figure 3a This is a schematic diagram of the encoding process as an example.

[0069] Figure 3b This is a schematic diagram illustrating the encoding process as an example.

[0070] Figure 3c This is a schematic diagram of the structure of an exemplary encoding system;

[0071] Figure 3d This is a schematic diagram illustrating the encoding process as an example.

[0072] Figure 4a This is a schematic diagram illustrating the decoding process as an example.

[0073] Figure 4b This is a schematic diagram illustrating the decoding process as an example.

[0074] Figure 4c This is a schematic diagram of the structure of an exemplary decoding system;

[0075] Figure 4d This is a schematic diagram illustrating the decoding process as an example.

[0076] Figure 5a This is a schematic diagram illustrating the encoding process as an example.

[0077] Figure 5b This is a schematic diagram of the structure of a subnetwork, as exemplarily shown.

[0078] Figure 5c This is a schematic diagram illustrating the decoding process as an example.

[0079] Figure 6 This is a schematic diagram of the encoding process as an example.

[0080] Figure 7 This is a schematic diagram illustrating the decoding process as an example.

[0081] Figure 8This is a schematic diagram of the encoding process as an example.

[0082] Figure 9 This is a schematic diagram illustrating the decoding process as an example.

[0083] Figure 10 This is a schematic diagram of the encoding process as an example.

[0084] Figure 11 This is a schematic diagram illustrating the decoding process as an example.

[0085] Figure 12a This is an illustrative diagram showing the compression performance effect as an example.

[0086] Figure 12b This is an example illustration of image quality.

[0087] Figure 13 This is a schematic diagram of the structure of an exemplary device. Detailed Implementation

[0088] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0089] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0090] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0091] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0092] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0093] Figure 1 The diagram illustrates an application scenario as an example.

[0094] Reference Figure 1 For example, the image compression process can be as follows: AI encoding can be performed on the frame to be encoded to obtain intermediate data, which can refer to the data to be transmitted to the decoding end; for example, it can include inter-frame edge information (which can refer to the relative motion information of each pixel of the frame to be encoded relative to the previous frame), residual information (the difference between the frame to be decoded and the corresponding prediction information), etc., which are not limited in this application. Next, entropy encoding can be performed on the intermediate data to obtain the bitstream of the frame to be encoded.

[0095] Continue to refer to Figure 1 For example, the image decompression process can be as follows: after obtaining the bitstream, the bitstream can be entropy decoded to obtain the corresponding intermediate data; then, AI decoding is performed based on the intermediate data to obtain the corresponding reconstructed image.

[0096] For example, entropy coding refers to coding that does not lose any information according to the entropy principle during the coding process. Entropy coding can include various types, such as Shannon coding, Huffman coding, and arithmetic coding, etc. This application does not limit these types.

[0097] For example, the frame to be encoded can be any one of RAW (raw) image, RGB (a color mode, where "R" represents red, "G" represents green, and "B" represents blue) image, and YUV (a color mode, where "Y" represents luminance, "U" and "V" represent chrominance, "Chroma") image, and this application does not limit it.

[0098] For example, the compression and decompression processes can be performed in the same electronic device or in different electronic devices, and this application does not limit this.

[0099] For example, this application can be applied to compressing and decompressing a single image, or to compressing and decompressing multiple frames of images in a video sequence; this application does not limit the scope of the application.

[0100] For example, this application can be applied to a variety of scenarios, such as Huawei Cloud storage (or transmission) of images (or videos), video surveillance, live streaming, etc. This application does not limit these applications.

[0101] It should be understood that, Figure 1 The encoding process described above is merely an example of the encoding process in this application; the encoding process in this application may have more advanced features. Figure 1 The further steps shown are not limited to those described herein. Similarly, Figure 1 The decoding process described in the example is merely one example of the decoding process in this application; the decoding process in this application may have more advanced features. Figure 1 The further steps shown are not limited to those described herein. This application uses an encoding process including AI encoding and entropy encoding, and a decoding process including AI decoding and entropy decoding as an example for illustration.

[0102] For example, this application can employ an encoding network to implement the above. Figure 1 The AI ​​encoding and entropy encoding shown refer to the process where the frame to be encoded is input into the encoding network, where the encoding network performs AI encoding and entropy encoding, and outputs a bitstream.

[0103] For example, this application can also use a decoding network to implement the above. Figure 1 The AI ​​decoding and entropy decoding shown involve inputting the bitstream into a decoding network, which then performs entropy decoding and AI decoding to output a reconstructed image.

[0104] For example, the encoding and decoding networks can be pre-trained using different training methods, resulting in M ​​sets of network parameters (M being an integer greater than 1) for each network. The encoding and decoding networks are jointly trained, and thus, the M sets of network parameters for the encoding network and the M sets of network parameters for the decoding network are in one-to-one correspondence. The M sets of network parameters for the encoding network correspond to different compression performance information, and the M sets of network parameters for the decoding network also correspond to different compression performance information; this compression performance information can include image quality and / or compression ratio. In this way, during the encoding process of adjacent frames, using different network parameters results in alternating high and low compression performance between adjacent frames, constructing a temporal compression performance adversarial relationship, thereby structurally reducing accumulated error and improving compression performance.

[0105] For example, for ease of description, a target network can be used to represent the encoding network and the decoding network, illustrating the joint process of the encoding and decoding networks. For instance, the decoding network in the target network is connected after the encoding network; that is, the input of the target network is the input of the encoding network, the output of the encoding network is connected to the input of the decoding network, and the output of the decoding network is the output of the target network.

[0106] It should be noted that the frame used for encoding and decoding based on the reconstructed image of the previous frame is a P-frame. Therefore, the encoding network used to encode P-frames and the decoding network used to decode P-frames are as follows.

[0107] Figure 2 This is a schematic diagram illustrating the training process as an example.

[0108] S201, Obtain training images.

[0109] For example, multiple video sequences can be collected, each video sequence may include multiple frames of images, and a video sequence can be used as a set of training images; in this way, multiple sets of training images can be obtained, each set of training images may include multiple frames of images.

[0110] S202, Obtain the target network, which includes M pairs of initial network parameter sets, where M is an integer greater than 1.

[0111] For example, the encoding network may include M sets of network parameters, the decoding network may include M sets of network parameters, and the target network may include M sets of network parameters. For ease of description, the set of network parameters of the target network before training can be referred to as the default set of network parameters.

[0112] For example, multiple sets of training images can be used to perform the first stage of training on the target network to update the parameters in the default network parameter set, so as to obtain the initial network parameter set.

[0113] For example, the first stage of training is illustrated below using a set of training images. For example, consecutive X1 (X1 is a positive integer greater than 1) frames can be selected from this set of training images, where the first frame in the consecutive X1 frames is an I-frame (Intra-coded frame), and the other frames are P-frames. For example, the target network can first load the i-th (where i ranges from 1 to M, i is an integer) pair of default network parameter sets, and then input the reconstructed image of the I-frame from the consecutive X1 frames and the first P-frame into the target network to obtain the reconstructed image of the first P-frame output by the target network. After obtaining the reconstructed image of the first P-frame, the loss value is calculated based on the first P-frame and its reconstructed image to obtain the loss value corresponding to the first P-frame.

[0114] Next, the reconstructed image of the first P-frame and the second P-frame from the consecutive X1-frame images are input into the target network to obtain the reconstructed image of the second P-frame output by the target network. Then, based on the second P-frame and the reconstructed image of the second P-frame, the loss value is calculated to obtain the loss value corresponding to the second P-frame. By analogy, the loss values ​​corresponding to X1-1 P-frames can be obtained.

[0115] Subsequently, the loss values ​​corresponding to these X1-1 P frames can be combined to update the i-th pair of default network parameter sets of the target network. Then, using the above method, the i-th pair of default network parameter sets after the last update are updated sequentially using each group of training images until a first preset condition is met, thereby obtaining the i-th pair of initial network parameter sets. Here, the first preset condition can refer to the condition for stopping the first stage of training, and can be set according to requirements; this application does not impose any restrictions on it. Afterwards, the i-th pair of initial network parameter sets of the target network can be assigned to the other M-1 pairs of default network parameter sets of the target network; thus, M pairs of initial network parameter sets can be obtained. Here, the M pairs of initial network parameter sets are identical, that is, the M pairs of initial network parameter sets correspond to the same compression performance information.

[0116] Subsequently, multiple sets of training images are used to perform a second stage of training on the target network, updating the parameters in the M pairs of initial network parameter sets to obtain the M pairs of preset network parameter sets. The training process for the second stage can be referred to in S203:

[0117] S203, based on the M ways of determining the loss value and the training images, update the M pairs of initial network parameter sets of the target network respectively to obtain the M pairs of preset network parameter sets.

[0118] For example, using a method for determining the loss value and multiple sets of training images, the parameters in a pair of initial network parameter sets of the target network can be updated to obtain a pair of preset network parameter sets. These preset network parameter sets of the target network include: a preset network parameter set for the encoding network and a preset network parameter set for the decoding network. Thus, M preset network parameter sets for the encoding network and M preset network parameter sets for the decoding network can be obtained.

[0119] The following explanation uses updating the i-th pair of initial network parameters in M ​​pairs of initial network parameter sets as an example. Here, i ranges from 1 to M, and i is an integer.

[0120] For example, consider a set of training images. First, select X2 consecutive frames (X2 is a positive integer, and X2 can be greater than X1) from this set of training images. The first frame in the consecutive X2 frames is called an I-frame, and the other frames are called P-frames. The target network can first load the i-th pair of initial network parameter sets. Next, input the reconstructed image of the I-frame and the first P-frame from the consecutive X2 frames into the target network. The target network performs forward computation to obtain the reconstructed image of the first P-frame output by the target network. Subsequently, the loss value is calculated based on the first P-frame and its reconstructed image to obtain the loss value corresponding to the first P-frame.

[0121] Next, the reconstructed image of the first frame and the second P-frame from the consecutive X2 frames are input into the target network, which performs forward computation to obtain the reconstructed image of the second P-frame output by the target network. Then, based on the second P-frame and its reconstructed image, the loss value is calculated to obtain the loss value corresponding to the second P-frame. This process is repeated to obtain the loss values ​​corresponding to X2-1 P-frames.

[0122] For example, the i-th method of determining the loss value can be to combine the loss values ​​corresponding to the X2-1 P frames to determine the loss value used for backpropagation, which is the loss value corresponding to the i-th pair of initial network parameter sets. Then, the loss values ​​corresponding to the X2-1 P frames can be combined to perform backpropagation on the target network, that is, to update the parameters in the i-th pair of initial network parameter sets of the target network. Furthermore, using the above method, each set of training images is used sequentially to update the i-th pair of initial network parameter sets after the last update until the second preset condition is met, thereby obtaining the i-th pair of preset network parameter sets, that is, obtaining the i-th preset network parameter set of the encoding network and the i-th preset network set of the decoding network. The second preset condition can refer to the conditions for completing the training of the i-th pair of preset network parameter sets, and can be set according to requirements; this application does not impose any restrictions on this.

[0123] For example, assuming X2 = 4, the first 4 frames (i.e., IPPP) of this training image set can be used as input into the target network loaded with the i-th pair of initial network parameter sets, and the target network will perform forward computation. Then, the loss values ​​corresponding to the first P-frame, the second P-frame, and the third P-frame can be combined to update the i-th pair of initial network parameter sets of the target network to obtain the i-th pair of preset network parameter sets.

[0124] In one possible approach, the M determination methods correspond to M sets of computational data, which are used to calculate M sets of loss values. Each set of computational data includes at least one P-frame and its reconstructed image. That is, when updating any two pairs of initial network parameter sets in the M pairs of initial network parameter sets, the computational data used to calculate the loss values ​​is different. Thus, any two of the M preset network parameter sets in the resulting encoding network are different, corresponding to different compression performance information; and any two of the M preset network parameter sets in the decoding network are also different, corresponding to different compression performance information.

[0125] For example, for the j-th pair of initial network parameter sets (j ranges from 1 to M, j is an integer, j is not equal to i), the number of images selected from each training image set is X2, which is the same as the number of images selected from each training image set for the i-th pair of initial network parameter sets, but only the loss values ​​corresponding to X2-2 P frames (excluding the first P frame) are combined for updating.

[0126] Suppose that for the j-th pair of initial network parameter sets, the number of images selected from each training image group is X2, which is the same as the number of training images selected from each training image group for the i-th pair of initial network parameter sets, both being equal to 4. Then, the first 4 frames (i.e., IPPP) of this training image group can be used as input into the target network loaded with the j-th pair of initial network parameter sets, and the target network will perform forward calculation. However, when performing backpropagation on the target network, only the loss value corresponding to the second P-frame and the loss value corresponding to the third P-frame are combined to update the j-th pair of initial network parameter sets of the target network.

[0127] For example, the number of images selected from each training image set for the j-th pair of initial network parameters is X2, which is different from the number of images selected from each training image set for the i-th pair of initial network parameters; and the P-frames used to calculate the corresponding loss value for the j-th pair of initial network parameters are different from the P-frames used to calculate the corresponding loss value for the i-th pair of initial network parameters.

[0128] Suppose that for the i-th pair of initial network parameters, the number of images selected from each training image group is X2 = 4, and for the j-th pair of initial network parameters, the number of images selected from each training image group is X2 = 5. Then, the first 5 frames (i.e., IPPPP) of that training image group can be used to update the j-th pair of initial network parameters. Specifically, the loss values ​​corresponding to the 1st P-frame, the 2nd P-frame, the 3rd P-frame, and the 4th P-frame can be combined to update the j-th pair of initial network parameters of the target network.

[0129] It should be noted that, for the initial network parameter set M, the better the compression performance information of the corresponding preset network parameter set, the more loss values ​​corresponding to the more P frames are integrated into the initial network parameter set during the update process.

[0130] In one possible approach, the M determination methods correspond to M loss functions. That is, updating any two pairs of initial network parameter sets from the M pairs of initial network parameter sets uses different loss functions. For example, updating the i-th pair of initial network parameter sets uses loss function I; updating the j-th pair of initial network parameter sets (where j ranges from 1 to M, is an integer, and j is not equal to i) uses loss function J. Thus, any two of the M preset network parameter sets in the resulting encoding network are different, corresponding to different compression performance information; and any two of the M preset network parameter sets in the decoding network are also different, corresponding to different compression performance information.

[0131] For example, loss function I can be λ1*d+bpp, and loss function J can be λ2*d+bpp; λ1 is not equal to λ2. Here, the smaller d (distortion), the higher the image quality; and the larger bpp (bits per pixel), the lower the compression ratio. Thus, the loss value calculated using λ1*d+bpp can be used to update the i-th pair of initial network parameters to obtain the i-th pair of preset network parameters. Similarly, the loss value calculated using λ2*d+bpp can be used to update the j-th pair of initial network parameters to obtain the j-th pair of preset network parameters. If λ1 is less than λ2, the compression performance of the j-th pair of preset network parameters is higher than that of the i-th pair; if λ1 is greater than λ2, the compression performance of the i-th pair of preset network parameters is higher than that of the j-th pair.

[0132] It should be understood that for any two pairs of initial network parameter sets in the M pairs of initial network parameter sets, when the loss function used to calculate the loss value is different and / or the calculation data used to calculate the loss value is different, the two pairs of preset network parameter sets obtained will be different and will correspond to different compression performance information.

[0133] In one possible approach, the encoding network may include Y (Y is a positive integer) subnetworks, each of which includes M sets of parameters. Thus, in the resulting M preset network parameter sets of the encoding network, each preset network parameter set may include Y preset network parameter groups, each preset network parameter group corresponds to a subnetwork, and all preset network parameter groups of any two preset network parameter sets in the M preset network parameter sets are different.

[0134] In one possible approach, the decoding network may include Y subnetworks, each of which includes M sets of parameters. Thus, in the resulting set of M preset network parameters, each preset network parameter set may include Y preset network parameter groups, each preset network parameter group corresponding to a subnetwork, and all preset network parameter groups of any two preset network parameter sets in the M preset network parameter sets are different.

[0135] In one possible approach, the encoding network comprises Y sub-networks, where each of the Y1 sub-networks contains M sets of parameters, while the other Y2 sub-networks each contain only one set of parameters; where Y1 and Y2 are positive integers, and Y1 + Y2 = Y. Thus, in the M preset network parameter sets corresponding to the encoding network obtained using the above method, each preset network parameter set contains only Y1 preset network parameter sets. After the first stage of training, Y2 initial shared network parameter sets are obtained for the encoding network, each initial shared network parameter set corresponding to one of the Y2 sub-networks.

[0136] In one possible approach, the decoding network comprises Y sub-networks. Y1 sub-networks each contain M sets of parameters, while the other Y2 sub-networks each contain only one set of parameters. Thus, in the M preset network parameter sets corresponding to the decoding network obtained using the above method, each preset network parameter set contains only Y1 preset network parameter sets. After the first stage of training, Y2 initial shared network parameter sets are obtained for the decoding network, and each initial shared network parameter set corresponds to one of the Y2 sub-networks.

[0137] Thus, the target network can include Y2 pairs of initial shared network parameter sets. Each pair of initial shared network parameter sets can include: the initial shared network parameter set corresponding to a sub-network of the encoding network and the initial shared network parameter set corresponding to a sub-network of the decoding network. For example, the Y2 pairs of initial shared network parameter sets can be updated simultaneously with updating each pair of initial network parameter sets in the M pairs of initial network parameter sets; this yields the Y2 pairs of preset shared network parameter sets. The specific process is as follows:

[0138] For example, when loading the i-th pair of initial network parameter sets into the target network, the Y2-pair initial shared network parameter set is loaded; and when updating the i-th pair of initial network parameter sets according to the loss value corresponding to the i-th pair of initial network parameter sets, the Y2-pair initial shared network parameter set is updated using the i-th set of updates according to the loss value corresponding to the i-th pair of initial network parameter sets. Thus, while obtaining M pairs of preset network parameter sets, the Y2-pair initial shared network parameter set is updated using M sets of updates, thereby obtaining the Y2-pair preset shared network parameter set. Then, each of the Y2-pair preset shared network parameter sets can be added to the M pairs of preset network parameter sets. Each pair of preset shared network parameter sets in the Y2-pair preset shared network parameter set can include one preset shared network parameter set for the encoding network and one preset shared network parameter set for the decoding network. Thus, each of the Y2 preset shared network parameter sets of the encoding network can be added to the M preset network parameter sets of the encoding network; and each of the Y2 preset shared network parameter sets of the decoding network can be added to the M preset network parameter sets of the decoding network.

[0139] Thus, in the M preset network parameter sets of the encoding network, any two preset network parameter sets have the same Y2 preset shared network parameter groups and different Y1 preset network parameter groups. Correspondingly, in the M preset network parameter sets of the decoding network, any two preset network parameter sets have the same Y2 preset shared network parameter groups and different Y1 preset network parameter groups.

[0140] Through the above training, we can obtain M preset network parameters for the encoding network and M preset network parameters for the decoding network.

[0141] The following describes the encoding and decoding process of the frame to be encoded and decoded using the corresponding preset network parameter set.

[0142] Figure 3a This is a schematic diagram illustrating the encoding process as an example.

[0143] S301, Obtain the frame to be encoded.

[0144] For example, after acquiring the frame to be encoded, it is determined whether the frame to be encoded is a P-frame. If the frame to be encoded is a P-frame, steps S302 to S303 can be executed to encode the frame. If the frame to be encoded is an I-frame, the I-frame can be input to an I-frame encoding network, which encodes the I-frame and outputs the bitstream of the frame to be encoded. Here, the I-frame encoding network can refer to a network used for encoding I-frames.

[0145] S302, determine the network parameter set corresponding to the frame to be encoded from M preset network parameter sets. The M preset network parameter sets correspond to different compression performance information, and M is an integer greater than 1.

[0146] For example, network parameter selection conditions can be preset. For instance, the network parameter selection conditions can be set according to the frame identifier of the frame to be encoded, or the compression performance information corresponding to the M preset network parameter sets, or the network parameter sets corresponding to the previous few encoded frames of the frame to be encoded. The details will be explained later.

[0147] For example, when the frame to be encoded is a P-frame, the network parameter set corresponding to the frame to be encoded can be selected from M preset network parameter sets according to the network parameter selection conditions.

[0148] S303: The encoding network encodes the frame to be encoded according to the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded.

[0149] For example, the encoding network can load the set of network parameters corresponding to the frame to be encoded, and then use the reconstructed image of the previous frame as a reference to encode the frame to be encoded, thereby obtaining the bitstream of the frame to be encoded. The bitstream can then be sent to other devices or stored locally, and this application does not impose any restrictions on this.

[0150] It should be noted that in one possible approach, S301 to S303 are all executed by the encoding network.

[0151] Figure 3b This is a schematic diagram illustrating the encoding process as an example.

[0152] Suppose that the video sequence to be encoded consists of 5 frames, and these 5 frames are IPPPP.

[0153] Reference Figure 3b For example, at time t0, the first frame (i.e., the I-frame) in the video sequence to be encoded is the frame to be encoded. The I-frame can be input into the I-frame encoding network, where the I-frame encoding network can refer to a network used to encode the I-frame. Then, after processing by the I-frame encoding network, on the one hand, the I-frame encoding network can output the bitstream of the I-frame; on the other hand, the I-frame encoding network can output the reconstructed image of the I-frame, and use the reconstructed image of the I-frame as the input to the P-frame encoding network at the next time step (i.e., time t1). Here, the P-frame encoding network can refer to a network used to encode the P-frame, which is the encoding network described in this application.

[0154] At time t1, the second frame (the first P-frame) in the video sequence to be encoded is the frame to be encoded. If the P-frame encoding network determines that the network parameter set corresponding to the frame to be encoded is a preset network parameter set 1, then the P-frame encoding network can load the preset network parameter set 1; and it can input the reconstructed image of the I-frame output by the I-frame encoding network and the first P-frame into the P-frame encoding network. Then, after processing by the P-frame encoding network, on the one hand, the P-frame encoding network can output the bitstream of the first P-frame; on the other hand, the P-frame encoding network can use the reconstructed image of the first P-frame as the input of the P-frame encoding network at the next time step (time t2).

[0155] At time t2, the third frame (the second P-frame) in the video sequence to be encoded is the frame to be encoded. If the P-frame encoding network determines that the network parameter set corresponding to the frame to be encoded is a preset network parameter set 2, then the P-frame encoding network can load the preset network parameter set 2; and it can input the reconstructed image of the first P-frame output by the P-frame encoding network at time t1 and the second P-frame into the P-frame encoding network. Then, after processing by the P-frame encoding network, on the one hand, the P-frame encoding network can output the bitstream of the second P-frame; on the other hand, the P-frame encoding network can use the reconstructed image of the second P-frame as the input to the P-frame encoding network at the next time step. And so on, through the above method, the encoding of the video sequence to be encoded can be completed, obtaining the bitstream of the video sequence to be encoded.

[0156] Figure 3c This is a schematic diagram of the structure of an exemplary encoding system.

[0157] Reference Figure 3c For example, the encoding system may include an image acquisition module, a selector, and an encoding network. The image acquisition module performs step S301, the selector performs step S302, and the encoding network performs step S303.

[0158] Figure 3d This is a schematic diagram illustrating the encoding process as an example.

[0159] Figure 3d The encoding process and Figure 3b The difference in the encoding process is that, Figure 3dThe set of network parameters corresponding to the frame to be encoded loaded by the P-frame encoding network at each time step is determined by the selector. At time t1, the selector determines the set of network parameters corresponding to the frame to be encoded as preset parameter set 1 from M preset network parameter sets; then, the set identifier of preset parameter set 1 / preset parameter set 1 can be sent to the P-frame encoding network; thus, the P-frame encoding network can load preset parameter set 1. At time t2, the selector determines the set of network parameters corresponding to the frame to be encoded as preset parameter set 2 from M preset network parameter sets; then, the set identifier of preset parameter set 2 / preset parameter set 2 can be sent to the P-frame encoding network; thus, the P-frame encoding network can load preset parameter set 2; and so on. Figure 3d Other steps in the encoding process and Figure 3b The other steps in the encoding process are similar and will not be described in detail here.

[0160] In this way, the encoding network can use different sets of network parameters during the encoding of multiple adjacent frames. Since the compression performance information corresponding to each preset set of network parameters is different, the compression performance of multiple adjacent frames can be alternated between high and low, which constructs compression performance adversarial in time sequence, thereby structurally reducing the cumulative error and improving compression performance.

[0161] Figure 4a This is a schematic diagram illustrating the decoding process as an example.

[0162] S401, obtain the bitstream of the frame to be decoded.

[0163] For example, after obtaining the bitstream of the frame to be decoded, the bitstream can be parsed to determine whether the frame to be decoded is a P-frame. If the frame to be decoded is a P-frame, steps S402 to S403 can be executed to decode the bitstream of the frame to be decoded. If the frame to be decoded is an I-frame, the bitstream of the I-frame can be input to the I-frame decoding network, which decodes the bitstream of the I-frame to obtain the reconstructed image. Here, the I-frame decoding network can refer to a network used for decoding I-frames.

[0164] S402, determine the network parameter set corresponding to the frame to be decoded from M preset network parameter sets, where the M preset network parameter sets correspond to different compression performance information, and M is an integer greater than 1.

[0165] In one possible approach, after the encoding end obtains the bitstream of the frame to be encoded, it can add a set identifier of the network parameter set to the bitstream of the frame to be encoded. In this way, after the decoding end receives the bitstream, it can determine the network parameter set corresponding to the frame to be decoded from M preset network parameter sets by using the set identifier parsed from the bitstream.

[0166] In one possible approach, the decoding and encoding ends pre-share the network parameter selection criteria. This way, the encoding end, after receiving the bitstream of the frame to be encoded, does not need to modify the bitstream. Furthermore, the decoding end, upon receiving the bitstream, can directly determine the network parameter set corresponding to the frame to be decoded from M preset network parameter sets according to the network parameter selection criteria.

[0167] S403: The decoding network decodes the bitstream of the frame to be decoded according to the set of network parameters corresponding to the frame to be decoded, so as to obtain the reconstructed image of the frame to be decoded.

[0168] For example, the decoding network can load the set of network parameters corresponding to the bitstream of the frame to be decoded, and then decode the bitstream of the frame to be decoded with reference to the reconstructed image of the previous frame to obtain the reconstructed image of the frame to be decoded. Then, the decoding end can display the decoded reconstructed image.

[0169] It should be noted that in one possible approach, S401 to S403 are all executed by the decoding network.

[0170] Figure 4b This is a schematic diagram illustrating the decoding process as an example.

[0171] Assume the bitstream to be decoded consists of 5 frames, each representing an IP / PPP frame. (Refer to...) Figure 4b For example, at time t0, the bitstream of the first frame in the bitstream to be decoded (i.e., the bitstream of the I-frame) is the bitstream of the frame to be decoded. The bitstream of the I-frame can be input into the I-frame decoding network, which decodes it and outputs the reconstructed image of the I-frame. The I-frame decoding network can refer to a network used for decoding I-frames. Then, on one hand, the I-frame decoding network can send the reconstructed image of the I-frame to the display module for display. On the other hand, the I-frame decoding network can use the reconstructed image of the I-frame as input to the P-frame decoding network at the next time step (time t1). The P-frame decoding network can refer to a network used for decoding P-frames, which is the decoding network described in this application.

[0172] At time t1, the bitstream of the second frame in the bitstream to be decoded (that is, the bitstream of the first P-frame) is the bitstream of the frame to be decoded. If the P-frame encoding network determines that the network parameter set corresponding to the frame to be decoded is a preset network parameter set 1, then the P-frame encoding network can load the preset network parameter set 1; and can input the reconstructed image of the I-frame output by the I-frame decoding network and the bitstream of the first P-frame into the P-frame decoding network, which will then decode and output the reconstructed image of the first P-frame. On one hand, the P-frame decoding network can send the reconstructed image of the first P-frame to the display module for display. On the other hand, the P-frame decoding network can use the reconstructed image of the first P-frame as the input to the P-frame decoding network at the next time (t2).

[0173] At time t2, the bitstream of the third frame (i.e., the bitstream of the second P-frame) in the bitstream to be decoded is the bitstream of the frame to be decoded. If the P-frame encoding network determines that the network parameter set corresponding to the frame to be decoded is a preset network parameter set 2, then the P-frame encoding network can load the preset network parameter set 2; and it can input the reconstructed image of the first P-frame output by the P-frame decoding network at time t1 and the bitstream of the second P-frame into the P-frame decoding network, which will then decode and output the reconstructed image of the second P-frame. On one hand, the P-frame decoding network can send the reconstructed image of the second P-frame to the display module for display. On the other hand, the P-frame decoding network can use the reconstructed image of the second P-frame as the input to the P-frame decoding network at the next time step. And so on, the decoder can complete the decoding of the bitstream to be decoded.

[0174] Figure 4c This is a schematic diagram of the structure of an exemplary decoding system.

[0175] Reference Figure 4c For example, the decoding system may include a bitstream acquisition module, a selector, and a decoding network. The bitstream acquisition module performs S401, the selector performs S402, and the decoding network performs S403.

[0176] Figure 4d This is a schematic diagram illustrating the decoding process as an example.

[0177] Figure 4d The decoding process and Figure 4b The difference in the decoding process is that, Figure 4dThe set of network parameters corresponding to the frame to be decoded loaded by the P-frame decoding network at each time step is determined by the selector. At time t1, the selector determines the set of network parameters corresponding to the frame to be decoded as preset parameter set 1 from M preset network parameter sets; then, the set identifier of preset parameter set 1 / preset parameter set 1 can be sent to the P-frame decoding network; thus, the P-frame decoding network can load preset parameter set 1. At time t2, the selector determines the set of network parameters corresponding to the frame to be decoded as preset parameter set 2 from M preset network parameter sets; then, the set identifier of preset parameter set 2 / preset parameter set 2 can be sent to the P-frame decoding network; thus, the P-frame decoding network can load preset parameter set 2; and so on. Figure 4d Other steps in the decoding process and Figure 4b The other steps in the decoding process are similar and will not be described in detail here.

[0178] Figure 5a This is a schematic diagram illustrating the encoding process as an example.

[0179] For example, the coding network may include AI coding units and entropy coding units (such as...). Figure 5a As shown), the AI ​​coding unit is used for AI coding, and the entropy coding unit is used for entropy coding. For example, the AI ​​coding unit may include: an inter-frame transform network, an inverse inter-frame transform network, a prediction network, and a residual transform network, such as... Figure 5a As shown. It should be understood that, Figure 5a This is merely an example of the AI ​​coding unit in this application. The AI ​​coding unit in this application may have more or fewer networks than the network shown in Figure 5, such as a quantization network (for quantization), a probability estimation network (for probability estimation), etc., and this application does not impose any limitations on this. Furthermore, it should be understood that the coding network may also include other units, and this application does not impose any limitations on this. This application uses an example where the coding network includes an AI coding unit and an entropy coding unit, and the AI ​​coding unit includes: an inter-frame transform network, an inverse inter-frame transform network, a prediction network, and a residual transform network, for illustrative purposes.

[0180] For example, the inter-frame transform network includes R1 sub-networks, the inverse inter-frame transform network includes R2 sub-networks, the prediction network includes R3 sub-networks, and the residual transform network may include R4 sub-networks. The sum of R1, R2, R3, and R4 is less than or equal to Y; R1, R2, R3, and R4 are all positive integers, which can be set according to requirements, and this application does not impose any restrictions on them. Each of the M preset network parameter sets of the coding network may include: R1 preset network parameter set 1, R2 preset network parameter set 2, R3 preset network parameter set 3, and R4 preset network parameter set 4. The R1 preset network parameter set 1 corresponds to the R1 sub-networks of the inter-frame transform network, the R2 preset network parameter set 2 corresponds to the R2 sub-networks of the inverse inter-frame transform network, the R3 preset network parameter set 3 corresponds to the R3 sub-networks of the prediction network, and the R4 preset network parameter set 4 corresponds to the R4 sub-networks of the residual transform network.

[0181] It should be noted that each sub-network can be composed of network layers such as convolutional layers, downsampling layers, and activation layers, and this application does not impose any restrictions on this.

[0182] Figure 5b This is a schematic diagram illustrating the structure of a subnetwork as an example. Figure 5b The neutron network can include 3 convolutional layers and 3 activation layers, which can be connected in the order of "convolutional layer → activation layer → convolutional layer → activation layer → convolutional layer → activation layer". Here, "(W×5×5) / 2↓" indicates that the convolutional layer can include W channels, each convolutional kernel is 5×5 in size, and convolution is performed with a stride of 2.

[0183] It should be understood that, Figure 5b This is merely one example of a subnetwork in this application; the subnetworks in this application may include more than [examples of subnetworks]. Figure 5b This application does not limit the number of network layers, whether more or fewer.

[0184] In one possible approach, any one of the Y subnetworks of the encoding network has a different set of preset network parameters in any two of the M preset network parameter sets.

[0185] For example, the preset network parameter group 1 corresponding to each of the R1 subnetworks in the inter-frame transform network is different from the preset network parameter group 1 corresponding to the preset network parameter set 1 in the preset network parameter set 2.

[0186] For example, in the R2 subnetworks of the inter-frame inverse transform network, the corresponding preset network parameter group 2 in preset network parameter set 1 is different from the corresponding preset network parameter group 2 in preset network parameter set 2.

[0187] For example, the preset network parameter group 3 corresponding to each of the R3 subnetworks in the prediction network is different from the preset network parameter group 3 corresponding to the preset network parameter set 1.

[0188] For example, the preset network parameter group 4 corresponding to each of the R4 subnetworks in the residual transform network is different from the preset network parameter group 4 corresponding to the preset network parameter set 1.

[0189] In one possible approach, the Y1 subnetworks of the encoding network have different preset network parameter sets in any two of the M preset network parameter sets; and the other Y2 subnetworks of the encoding network have the same preset network parameter sets in any two of the M preset network parameter sets.

[0190] For example, the G1 sub-networks of the inter-frame transform network have different preset network parameter groups 1 in preset network parameter set 1 and preset network parameter groups 1 in preset network parameter set 2. The other G2 sub-networks of the inter-frame transform network have the same preset network parameter groups 1 in preset network parameter set 1 and preset network parameter groups 1 in preset network parameter set 2. Here, G1 + G2 = R1, and G1 and G2 are integers that can be set according to requirements; this application does not impose any restrictions on this.

[0191] For example, the G3 sub-networks of the inter-frame inverse transform network have different preset network parameter groups 2 in preset network parameter set 1 and 2 in preset network parameter set 2. The other G4 sub-networks of the inter-frame inverse transform network have the same preset network parameter groups 2 in preset network parameter set 1 and 2 in preset network parameter set 2. Here, G3 + G4 = R2, and G3 and G4 are integers that can be set according to requirements; this application does not impose any restrictions on this.

[0192] For example, the G5 subnetworks of the prediction network have different preset network parameter groups 3 in preset network parameter set 1 and in preset network parameter set 2. The other G6 subnetworks of the prediction network have the same preset network parameter groups 3 in preset network parameter set 1 and in preset network parameter set 2. Here, G5 + G6 = R3, where G5 and G6 are integers and can be set according to requirements; this application does not impose any restrictions on this.

[0193] For example, the G7 subnetworks of the residual transform network have different preset network parameter groups 4 in preset network parameter set 1 and preset network parameter groups 4 in preset network parameter set 2. The other G8 subnetworks of the residual transform network have the same preset network parameter groups 4 in preset network parameter set 1 and preset network parameter groups 4 in preset network parameter set 2. Here, G7 + G8 = R4, where G7 and G8 are integers and can be set according to requirements; this application does not impose any restrictions on this.

[0194] Among them, the sum of G1, G3, G5 and G7 is less than or equal to Y1; the sum of G2, G4, G6 and G8 is less than or equal to Y2.

[0195] Reference Figure 5a For example, after the encoding end obtains the frame to be encoded, it can input the reconstructed images of the frame to be encoded and its reference frame (which could be the previous frame) into the inter-frame transform network. The inter-frame transform network processes these images and outputs the inter-frame side information to be compressed. Then, on one hand, the entropy coding unit performs entropy coding on the inter-frame side information to be compressed, obtaining the bitstream of the inter-frame side information. On the other hand, the inter-frame side information to be compressed can be input into the inverse inter-frame transform network, which processes it and outputs the inter-frame side information.

[0196] Subsequently, inter-frame edge information and the reconstructed image of the reference frame of the frame to be encoded can be input into the prediction network, which processes the information and outputs prediction information. Then, the difference between the reconstructed image of the reference frame and the prediction information is calculated to obtain residual information. This residual information is then input into the residual transform network, which processes it and outputs the residual information to be compressed. Finally, the entropy coding unit entropy codes the residual information to be compressed, obtaining the bitstream of the residual information.

[0197] The bitstream of the inter-frame side information and the bitstream of the residual information together form the bitstream of the frame to be encoded; then the encoding end can send the bitstream of the frame to be encoded to the decoding end.

[0198] Figure 5c This is a schematic diagram illustrating the decoding process as an example.

[0199] For example, the decoding network may include an AI decoding unit and an entropy decoding unit (such as...). Figure 5c As shown in the diagram, the AI ​​decoding unit is used for AI decoding, and the entropy decoding unit is used for entropy decoding. For example, the AI ​​decoding unit may include: an inter-frame inverse transform network, a prediction network, and a residual inverse transform network, such as... Figure 5c As shown. It should be understood that, Figure 5cThis is merely an example of the AI ​​decoding unit in this application. The AI ​​decoding unit of this application may have more or fewer networks than the network shown in Figure 5, such as an inverse quantization network (for inverse quantization), a probability estimation network (for probability estimation), etc., and this application is not limited thereto. Furthermore, it should be understood that the decoding network may also include other units, and this application is not limited thereto either. This application uses an example where the decoding network includes an AI decoding unit and an entropy decoding unit, and the AI ​​decoding unit includes an inter-frame inverse transform network, a prediction network, and a residual inverse transform network, for illustrative purposes.

[0200] For example, the inter-frame inverse transform network includes R2 sub-networks, the residual inverse transform network includes H1 sub-networks, and the prediction network includes R3 sub-networks. The sum of R2, H1, and R3 is less than or equal to Y; R2, H1, and R3 are all positive integers, and can be set according to requirements; this application does not impose any restrictions on this. Each of the M preset network parameter sets of the decoding network includes: R2 preset network parameter sets 2, H1 preset network parameter sets 5, and R3 preset network parameter sets 3. The R2 preset network parameter sets 2 correspond to the R2 sub-networks of the inter-frame inverse transform network, the H1 preset network parameter sets 5 correspond to the H1 sub-networks of the residual inverse transform network, and the R3 preset network parameter sets 3 correspond to the R3 sub-networks of the prediction network.

[0201] In one possible approach, any one of the Y subnetworks of the decoding network has a different set of preset network parameters in any two of the M preset network parameter sets.

[0202] For example, in the R2 subnetworks of the inter-frame inverse transform network, the corresponding preset network parameter group 2 in preset network parameter set 1 is different from the corresponding preset network parameter group 2 in preset network parameter set 2.

[0203] For example, the preset network parameter group 5 corresponding to each of the H1 subnetworks in the residual inverse transform network is different from the preset network parameter group 5 corresponding to the preset network parameter set 1.

[0204] For example, the preset network parameter group 3 corresponding to each of the R3 subnetworks in the prediction network is different from the preset network parameter group 3 corresponding to the preset network parameter set 1.

[0205] In one possible approach, the Y1 subnetworks of the decoding network have different preset network parameter sets in any two of the M preset network parameter sets; and the other Y2 subnetworks of the decoding network have the same preset network parameter sets in any two of the M preset network parameter sets.

[0206] For example, the G3 subnetworks of the inter-frame inverse transform network have different preset network parameter groups 2 and 3 in preset network parameter set 1. The other G4 subnetworks of the inter-frame inverse transform network have the same preset network parameter group 2 in preset network parameter set 1 and preset network parameter group 2 in preset network parameter set 2.

[0207] For example, the F1 subnetworks of the residual inverse transform network have different preset network parameter sets 5 in preset network parameter set 1 and 2. The other F2 subnetworks of the residual inverse transform network have the same preset network parameter sets 5 in preset network parameter set 1 and 2. Here, F1 + F2 = H1, and F1 and F2 are integers that can be set according to requirements; this application does not impose any restrictions on this.

[0208] For example, the G5 subnetworks of the prediction network have different preset network parameter groups 3 in preset network parameter set 1 and in preset network parameter set 2. The other G6 subnetworks of the prediction network have the same preset network parameter groups 3 in preset network parameter set 1 and in preset network parameter set 2.

[0209] Among them, the sum of G3, F1 and G5 is less than or equal to Y1; the sum of G4, F2 and GF6 is less than or equal to Y2.

[0210] Reference Figure 5c For example, after receiving the bitstream, the decoding end can perform entropy decoding on the bitstream using an entropy decoding unit to obtain the inter-frame edge information to be compressed and the residual information to be compressed. Next, on one hand, the inter-frame edge information to be compressed can be input into an inverse inter-frame transform network (IMTN) for processing, outputting the inter-frame edge information; then, the inter-frame edge information and the reconstructed image of the reference frame of the frame to be encoded are input into a prediction network for processing, outputting prediction information. On the other hand, the residual information to be compressed can be input into an inverse residual transform network (IRTN) for processing, outputting residual information. Finally, the residual information and the prediction information can be added together to obtain the reconstructed image.

[0211] The following describes the process of determining the set of network parameters corresponding to the frame to be encoded from M preset sets of network parameters.

[0212] Figure 6 This is a schematic diagram illustrating an exemplary encoding process. Figure 6 In one embodiment, the encoding network can encode N consecutive P frames (N being an integer greater than 1) using the same set of network parameters.

[0213] S601, Obtain the frame to be encoded.

[0214] For example, after obtaining the frame to be encoded, if it is determined that the frame to be encoded is a P-frame, then S602 to S605 can be executed to encode the frame to be encoded.

[0215] S602, determine whether the network parameter sets corresponding to the first N frames of the frame to be encoded are the same.

[0216] For example, this application can encode N consecutive P frames using the same set of network parameters; specifically, N can be set according to requirements, such as N=3, and this application does not limit the value of N. Furthermore, it can be determined whether the network parameter sets corresponding to the previous N frames to be encoded are the same; when the network parameter sets corresponding to the previous N frames to be encoded are the same, S603 can be executed; when at least one frame in the network parameter set corresponding to the previous N frames to be encoded has a different network parameter set than the network parameter sets corresponding to other frames, S604 can be executed.

[0217] S603, according to preset rules, select one preset network parameter set from M preset network parameter sets, excluding the network parameter set corresponding to the previous frame of the frame to be encoded, as the network parameter set corresponding to the frame to be encoded.

[0218] For example, preset rules can be set in advance, and the specific rules can be set according to the requirements. This application does not limit this. For example, k(i) = k(i-1) + 1, where k(i) is the index of the network parameter set corresponding to the frame to be encoded, k(i-1) is the index of the network parameter set corresponding to the previous frame of the frame to be encoded, and the numbers of the M preset network parameter sets can be from 1 to M.

[0219] Furthermore, when the network parameter sets corresponding to the previous N frames of the frame to be encoded are the same, a preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be encoded can be selected from the M preset network parameter sets according to preset rules, and used as the network parameter set corresponding to the frame to be encoded.

[0220] For example, suppose M = 5, which includes 5 preset network parameter sets: preset network parameter set 1, preset network parameter set 2, preset network parameter set 3, preset network parameter set 4, and preset network parameter set 5. If N = 3, and the network parameter set corresponding to the first 3 frames of the frame to be encoded is preset network parameter set 2, then preset network parameter set 3 is selected as the network parameter set corresponding to the frame to be encoded.

[0221] S604, determine the network parameter set corresponding to the previous frame of the frame to be encoded as the network parameter set corresponding to the frame to be encoded.

[0222] For example, when at least one of the network parameter sets corresponding to the previous N frames of the frame to be encoded is different from the network parameter sets corresponding to other frames, the network parameter set corresponding to the previous frame of the frame to be encoded can be used as the network parameter set corresponding to the frame to be encoded.

[0223] For example, based on the above example, if the network parameter sets corresponding to the first 3 frames of the frame to be encoded are: preset network parameter set 1, preset network parameter set 2 and preset network parameter set 2, then preset network parameter set 2 can be determined as the network parameter set corresponding to the frame to be encoded.

[0224] S605, the encoding network encodes the frame to be encoded according to the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded.

[0225] For example, the AI ​​encoding unit in the encoding network can load the network parameter set corresponding to the frame to be encoded. Then, the reconstructed image of the previous frame and the frame to be encoded can be input into the AI ​​encoding unit for processing to obtain intermediate data. Then, the entropy encoding unit in the encoding network performs entropy encoding on the intermediate data to obtain the bitstream of the frame to be encoded.

[0226] Figure 7 This is a schematic diagram illustrating an exemplary decoding process. Figure 7 In the embodiments described, the relationship with Figure 6 The encoding process corresponds to the decoding process.

[0227] S701, obtain the bitstream of the frame to be decoded.

[0228] For example, after obtaining the bitstream of the frame to be decoded, the bitstream of the frame to be decoded can be parsed to determine whether the frame to be decoded is a P-frame. When the frame to be decoded is a P-frame, steps S702 to S705 can be executed to decode the bitstream of the frame to be decoded.

[0229] S702, determine whether the network parameter sets corresponding to the first N frames of the frame to be decoded are the same.

[0230] For example, the encoding end uses the same set of network parameters for encoding N consecutive P frames; therefore, after receiving the bitstream, the decoding end can determine whether the set of network parameters corresponding to the first N frames of the frame to be decoded is the same. When the set of network parameters corresponding to the first N frames of the frame to be decoded is the same, S703 can be executed; when at least one frame in the set of network parameters corresponding to the first N frames of the frame to be decoded has a different set of network parameters than the set of network parameters corresponding to other frames, S704 can be executed.

[0231] S703, according to preset rules, select one preset network parameter set from M preset network parameter sets, excluding the network parameter set corresponding to the previous frame of the frame to be decoded, and determine it as the network parameter set corresponding to the frame to be decoded.

[0232] For example, when it is determined that the network parameter sets of the previous N frames of the frame to be decoded are the same, a preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be decoded can be selected from M preset network parameter sets according to preset rules, and used as the network parameter set corresponding to the frame to be decoded.

[0233] For example, suppose M = 5, which includes 5 preset network parameter sets: preset network parameter set 1, preset network parameter set 2, preset network parameter set 3, preset network parameter set 4, and preset network parameter set 5. If N = 3, and the network parameter set of the first 3 frames of the frame to be decoded is preset network parameter set 2, then preset network parameter set 3 can be selected as the network parameter set corresponding to the frame to be decoded.

[0234] S704, determine the set of network parameters corresponding to the previous frame of the frame to be decoded as the set of network parameters corresponding to the frame to be decoded.

[0235] For example, if it is determined that among the network parameter sets corresponding to the previous N frames of the frame to be decoded, there is at least one frame whose network parameter set is different from the network parameter sets corresponding to other frames, the network parameter set corresponding to the previous frame of the frame to be decoded can be determined as the network parameter set corresponding to the frame to be decoded.

[0236] For example, based on the above example, if the network parameter sets corresponding to the first 3 frames of the frame to be decoded are: preset network parameter set 1, preset network parameter set 2 and preset network parameter set 2, then preset network parameter set 2 can be determined as the network parameter set corresponding to the frame to be decoded.

[0237] S705: The decoding network decodes the bitstream of the frame to be decoded based on the set of network parameters corresponding to the frame to be decoded, so as to obtain the reconstructed image of the frame to be decoded.

[0238] For example, the entropy decoding unit included in the decoding network can perform entropy decoding on the bitstream to be decoded to obtain intermediate data; then, the AI ​​decoding unit included in the decoding network loads the network parameter set corresponding to the frame to be decoded, and inputs the reconstructed image of the previous frame of the frame to be decoded and the intermediate data into the AI ​​decoding unit, which processes the data and outputs the reconstructed image of the frame to be decoded.

[0239] Figure 8 This is a schematic diagram illustrating an exemplary encoding process. Figure 8 In the embodiment, M=2, that is, the encoding network includes two preset network parameter sets, and the encoding network can use different network parameter sets to encode two adjacent P frames.

[0240] S801, Obtain the frame to be encoded.

[0241] For example, after acquiring the frame to be encoded, and after determining that the frame to be encoded is a P-frame, it can be determined whether the frame identifier of the frame to be encoded is odd or even. When it is determined that the frame identifier of the frame to be encoded is odd, S802 can be executed; when it is determined that the frame identifier of the frame to be encoded is even, S803 can be executed.

[0242] For example, the frame identifier of the frame to be encoded can be a POC (Picture Order Count) value or a frame number of the frame to be encoded. This application does not limit this.

[0243] S802, when the frame identifier of the frame to be encoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded.

[0244] S803, when the frame identifier of the frame to be encoded is an even number, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded.

[0245] For example, the encoding network may include two sets of prediction network parameters: a first preset set of network parameters and a second set of network parameters. When the frame identifier of the frame to be encoded is odd, the first preset set of network parameters can be determined as the set of network parameters corresponding to the frame to be encoded; when the frame identifier of the frame to be encoded is even, the second preset set of network parameters can be determined as the set of network parameters corresponding to the frame to be encoded.

[0246] It should be understood that when the frame identifier of the frame to be encoded is odd, the second preset network parameter set can also be determined as the network parameter set corresponding to the frame to be encoded; when the frame identifier of the frame to be encoded is even, the first preset network parameter set can also be determined as the network parameter set corresponding to the frame to be encoded. That is to say, this application does not limit whether the encoding network uses the first preset network parameter set to encode the frame to be encoded with an odd frame identifier, or uses the second preset network parameter set to encode the frame to be encoded with an odd frame identifier.

[0247] In one possible approach, the compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set.

[0248] In one possible approach, the compression performance information corresponding to the second preset network parameter set is higher than the compression performance information corresponding to the first preset network parameter set.

[0249] For example, when the compression performance information includes image quality, the image quality corresponding to the first preset network parameter set is higher than the image quality corresponding to the second preset network parameter set; or, the image quality corresponding to the second preset network parameter set is higher than the image quality corresponding to the first preset network parameter set.

[0250] For example, when the compression performance information includes compression ratio, the compression ratio corresponding to the first preset network parameter set is higher than the compression ratio corresponding to the second preset network parameter set; or, the compression ratio corresponding to the second preset network parameter set is higher than the compression ratio corresponding to the first preset network parameter set.

[0251] For example, when the compression performance information includes image quality and compression ratio, the image quality corresponding to the first preset network parameter set is higher than the image quality corresponding to the second preset network parameter set, and the compression ratio corresponding to the first preset network parameter set is higher than the compression ratio corresponding to the second preset network parameter set. Alternatively, the image quality corresponding to the second preset network parameter set is higher than the image quality corresponding to the first preset network parameter set, and the compression ratio corresponding to the second preset network parameter set is higher than the compression ratio corresponding to the first preset network parameter set.

[0252] S804 is a coding network that encodes the frame to be encoded based on the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded.

[0253] For example, S804 can be referred to the description of S605, which will not be repeated here.

[0254] Figure 9 This is a schematic diagram illustrating an exemplary decoding process. Figure 9 In the embodiments described, the relationship with Figure 8The decoding method corresponding to the encoding method is M=2, which means that the decoding network corresponds to two preset network parameter sets. The decoding network can use different network parameter sets to decode two adjacent P frames.

[0255] S901, obtain the bitstream of the frame to be decoded.

[0256] For example, after obtaining the bitstream of the frame to be decoded, the bitstream of the frame to be decoded can be parsed to obtain the frame identifier and frame type of the frame to be decoded. Then, when it is determined that the frame to be decoded is a P-frame and the frame identifier is odd, S902 is executed; when it is determined that the frame to be decoded is a P-frame and the frame identifier is even, S903 is executed.

[0257] S902, when the frame identifier of the frame to be decoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded.

[0258] S903, when the frame identifier of the frame to be decoded is an even number, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded.

[0259] For example, the decoding network may include two sets of prediction network parameters: a first preset set of network parameters and a second set of network parameters. When the frame identifier of the frame to be decoded is odd, the first preset set of network parameters can be determined as the set of network parameters corresponding to the frame to be decoded; when the frame identifier of the frame to be decoded is even, the second preset set of network parameters can be determined as the set of network parameters corresponding to the frame to be decoded.

[0260] It should be understood that when the frame identifier of the frame to be decoded is odd, the second preset network parameter set can also be determined as the network parameter set corresponding to the frame to be decoded; when the frame identifier of the frame to be decoded is even, the first preset network parameter set can also be determined as the network parameter set corresponding to the frame to be decoded. In other words, this application does not limit whether the decoding network uses the first preset network parameter set to decode the frame to be decoded with an odd frame identifier, or uses the second preset network parameter set to decode the frame to be decoded with an odd frame identifier; it simply needs to correspond to the encoding network.

[0261] In one possible approach, the compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set.

[0262] In one possible approach, the compression performance information corresponding to the second preset network parameter set is higher than the compression performance information corresponding to the first preset network parameter set.

[0263] S904: The decoding network decodes the bitstream of the frame to be decoded according to the set of network parameters corresponding to the frame to be decoded, so as to obtain the reconstructed image of the frame to be decoded.

[0264] For example, S904 can be referred to the description of S705, and will not be repeated here.

[0265] Figure 10 This is a schematic diagram illustrating an exemplary encoding process. Figure 10 In one embodiment, the encoding network can select a set of network parameters for encoding the frame to be encoded based on the compression performance information corresponding to each preset set of network parameters.

[0266] S1001, Obtain the frame to be encoded.

[0267] S1002, determine the compression performance information corresponding to each of the M preset network parameter sets.

[0268] The following example illustrates how to determine the compression performance information corresponding to the i-th preset network parameter set.

[0269] For reference Figure 5a For example, the encoding network loads the i-th preset network parameter set. Then, the reconstructed images of the frame to be encoded and its reference frame are input to the inter-frame transform network to obtain the inter-frame edge information to be compressed, output by the inter-frame transform network. Next, the inter-frame edge information to be compressed is input to the inverse inter-frame transform network to obtain the inter-frame edge information output by the inverse inter-frame transform network. Subsequently, the reconstructed image of the reference frame and the inter-frame edge information are input to the prediction network to obtain the preset information output by the prediction network. Afterward, residual information is determined based on the reconstructed image of the reference frame and the prediction information, and the residual information is input to the residual transform network to obtain the residual information to be compressed, output by the residual transform network. For example, the same device may include an encoding network and a decoding network, which can be referred to... Figure 5b In this way, the decoding network can load the i-th preset network parameter set, and then input the residual information to be compressed into the inverse residual transform network to obtain the residual information output by the inverse residual transform network. Next, reconstruction can be performed based on the prediction information and the residual information to obtain the reconstructed image. For example, the frame to be encoded can be compared with its reconstructed image to determine compression performance information (such as PSNR (Peak Signal-to-Noise Ratio), BBP, etc.), which is the compression performance information of the i-th preset network parameter set. In this way, compression performance information corresponding to M preset network parameter sets can be obtained.

[0270] S1003, based on the compression performance information corresponding to the M preset network parameter sets, determine the network parameter set corresponding to the frame to be encoded from the M preset network parameter sets.

[0271] For example, the encoding network can encode K consecutive P frames (K is an integer, and K and N may be equal or unequal; this application does not impose any restrictions on this) using the same set of network parameters with compression performance information below a threshold, and encode the subsequent K consecutive P frames using the same set of network parameters with compression performance information above a threshold. The threshold can be set as needed; this application does not impose any restrictions on this.

[0272] Furthermore, when the compression performance information corresponding to the first K frames of the frame to be encoded is higher than a threshold, a preset set of network parameters whose compression performance information is lower than the threshold is used as the network parameter set for the frame to be encoded. If the compression performance information corresponding to the first K frames of the frame to be encoded is lower than the threshold, then a preset set of network parameters whose compression performance information is higher than the threshold is used as the network parameter set for the frame to be encoded.

[0273] Otherwise, if the compression performance information of the previous frame of the frame to be encoded is lower than the threshold, then a preset set of network parameters whose compression performance information is lower than the threshold is used as the set of network parameters for the frame to be encoded; if the compression performance information of the previous frame of the frame to be encoded is higher than the threshold, then a preset set of network parameters whose compression performance information is higher than the threshold is used as the set of network parameters for the frame to be encoded.

[0274] For example, assuming M=5, there are 5 preset network parameter sets: preset network parameter set 1, preset network parameter set 2, preset network parameter set 3, preset network parameter set 4, and preset network parameter set 5. Among these, the compression performance information corresponding to preset network parameter set 1 and preset network parameter set 2 is greater than a threshold, while the compression performance information corresponding to preset network parameter set 3, preset network parameter set 4, and preset network parameter set 5 is less than a threshold. Assuming K=2, if the preset network parameter set corresponding to the first two frames of the frame to be encoded is both preset network parameter set 2, then preset network parameter set 3, preset network parameter set 4, or preset network parameter set 5 is selected as the network parameter set corresponding to the frame to be encoded. If the preset network parameter set corresponding to the first two frames of the frame to be encoded is preset network parameter set 5, then preset network parameter set 1 or preset network parameter set 2 is selected as the network parameter set corresponding to the frame to be encoded. If the preset network parameter sets corresponding to the first two frames of the frame to be encoded are preset network parameter set 2 and preset network parameter set 3 respectively, then preset network parameter set 3, preset network parameter set 4, or preset network parameter set 5 is selected as the network parameter set corresponding to the frame to be encoded. If the preset network parameter sets corresponding to the first two frames of the frame to be encoded are preset network parameter set 3 and preset network parameter set 1 respectively, then preset network parameter set 1 or preset network parameter set 2 is selected as the network parameter set corresponding to the frame to be encoded.

[0275] S1004, the encoding network encodes the frame to be encoded according to the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded.

[0276] For example, in S1002, the encoding network has loaded the network parameter set to encode the frame to be encoded, and obtained the information to be compressed in the intermediate data of the frame to be encoded (i.e., the information to be compressed in the inter-frame side information and the residual information). At this time, the encoding network can encode the frame to be encoded by loading the network parameter set, and entropy encoding the information to be compressed in the intermediate data to obtain the bitstream of the frame to be encoded.

[0277] For example, in the M preset network parameter sets corresponding to the encoding network, each preset network parameter set has a corresponding set identifier, and each set identifier is used to uniquely identify a preset network parameter set. After obtaining the bitstream of the frame to be encoded, the set identifier of the network parameter set can be added to the bitstream of the frame to be encoded, so that the decoding end can use the corresponding network parameter set for decoding.

[0278] Figure 11 This is a schematic diagram illustrating an exemplary decoding process. Figure 11 In the embodiments described, the relationship with Figure 10 The decoding process corresponding to the encoding process.

[0279] S1101, Obtain the bitstream of the frame to be decoded.

[0280] S1102, parse the set identifier from the bitstream of the frame to be decoded.

[0281] S1103, determine the preset network parameter set that matches the set identifier from the M preset network parameter sets as the network parameter set corresponding to the frame to be decoded.

[0282] For example, among the M preset network parameter sets corresponding to the decoding network, each preset network parameter set has a corresponding set identifier, and each set identifier is used to uniquely identify a preset network parameter set. After obtaining the bitstream of the frame to be decoded, the bitstream of the frame to be decoded can be parsed to obtain the set identifier. Then, from the M preset network parameter sets corresponding to the decoding network, the preset network parameter set that matches the set identifier is searched; and then the preset network parameter set that matches the set identifier is determined as the network parameter set corresponding to the frame to be decoded.

[0283] S1104, the encoding network encodes the frame to be encoded according to the set of network parameters corresponding to the frame to be decoded, so as to obtain the bitstream of the frame to be encoded.

[0284] For example, S1104 can be referred to the description of S705, and will not be repeated here.

[0285] It should be understood that, in Figure 6 In one embodiment, after obtaining the bitstream of the frame to be encoded, a set identifier of the network parameter set corresponding to the frame to be encoded can be added to the bitstream of the frame to be encoded; in this way, after the encoding end receives the bitstream of the frame to be decoded, it can perform decoding with reference to S1101 to S1104.

[0286] What should be decoded is, in Figure 8 In one embodiment, after obtaining the bitstream of the frame to be encoded, a set identifier of the network parameter set corresponding to the frame to be encoded can be added to the bitstream of the frame to be encoded; in this way, after the encoding end receives the bitstream of the frame to be decoded, it can perform decoding with reference to S1101 to S1104.

[0287] Figure 12a This is a schematic diagram illustrating the compression performance effect as an example.

[0288] Reference Figure 12a The solid line represents the compression performance curve of the prior art, and the dashed line represents the compression performance curve of this application. A higher PSNR indicates higher image quality; a larger bpp indicates a lower compression ratio. By comparison, this application achieves a higher PSNR at the same compression ratio; or, under the premise of the same PSNR, a smaller bpp.

[0289] Figure 12b This is an example illustration of image quality.

[0290] Reference Figure 12b S2 is the image quality curve corresponding to the prior art, and S1 is the image quality curve corresponding to this application. By comparison, the image quality of each frame of this application has adversarial characteristics and is of higher quality.

[0291] In one example, Figure 13 An illustrative block diagram of an embodiment of this application shows an apparatus 1300 that may include a processor 1301 and a transceiver / transceiver pin 1302, and optionally, a memory 1303.

[0292] The various components of device 1300 are coupled together via bus 1304, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 1304 in the figure.

[0293] Optionally, the memory 1303 can be used to store instructions from the foregoing method embodiments. The processor 1301 can be used to execute the instructions in the memory 1303, control the receive pin to receive signals, and control the transmit pin to transmit signals.

[0294] Device 1300 may be an electronic device or a chip of an electronic device in the above method embodiments.

[0295] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0296] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the above-described related method steps to implement the encoding / decoding method and training method in the above embodiment.

[0297] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the encoding / decoding method and training method in the above embodiment.

[0298] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the encoding / decoding method and training method in the above-described method embodiments.

[0299] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0300] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0301] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0302] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0303] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0304] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0305] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0306] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0307] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0308] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0309] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An encoding method, characterized in that, The method includes: Obtain the frame to be encoded, which is a P-frame; From M preset network parameter sets, determine the network parameter set corresponding to the frame to be encoded, wherein the M preset network parameter sets correspond to different compression performance information, and M is an integer greater than 1; The encoding network encodes the frame to be encoded according to the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded; In the encoding process, the encoding network uses different sets of network parameters to encode multiple adjacent P-frames, so that the compression performance of the multiple adjacent P-frames varies.

2. The method according to claim 1, characterized in that, The M equals 2, and the M preset network parameter sets include a first preset network parameter set and a second preset network parameter set; The step of determining the network parameter set corresponding to the frame to be encoded from M preset network parameter sets includes: When the frame identifier of the frame to be encoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded; When the frame identifier of the frame to be encoded is an even number, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded.

3. The method according to claim 2, characterized in that, The compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set. or, The compression performance information corresponding to the second preset network parameter set is higher than that corresponding to the first preset network parameter set.

4. The method according to claim 1, characterized in that, The step of determining the network parameter set corresponding to the frame to be encoded from M preset network parameter sets includes: When the network parameter sets corresponding to the previous N frames of the frame to be encoded are the same, according to the preset rules, one preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be encoded is selected as the network parameter set corresponding to the frame to be encoded. Otherwise, the set of network parameters corresponding to the previous frame of the frame to be encoded is determined as the set of network parameters corresponding to the frame to be encoded, where N is an integer greater than 1.

5. The method according to claim 1, characterized in that, The step of determining the network parameter set corresponding to the frame to be encoded from M preset network parameter sets includes: Determine the compression performance information corresponding to each of the M preset network parameter sets; Based on the compression performance information corresponding to the M preset network parameter sets, the network parameter set corresponding to the frame to be encoded is determined from the M preset network parameter sets.

6. The method according to claim 5, characterized in that, The step of determining the network parameter set corresponding to the frame to be encoded from the M preset network parameter sets based on the compression performance information corresponding to each of the M preset network parameter sets includes: When the compression performance information corresponding to the first K frames of the frame to be encoded is higher than the threshold, a preset network parameter set whose compression performance information is lower than the threshold from the M preset network parameter sets is taken as the network parameter set corresponding to the frame to be encoded; if the compression performance information corresponding to the first K frames of the frame to be encoded is lower than the threshold, a preset network parameter set whose compression performance information is higher than the threshold from the M preset network parameter sets is taken as the network parameter set corresponding to the frame to be encoded, where K is a positive integer; Otherwise, if the compression performance information corresponding to the previous frame of the frame to be encoded is lower than the threshold, then the set of preset network parameters whose compression performance information is lower than the threshold from the M preset network parameter sets is taken as the set of network parameters corresponding to the frame to be encoded; if the compression performance information corresponding to the previous frame of the frame to be encoded is higher than the threshold, then the set of preset network parameters whose compression performance information is higher than the threshold from the M preset network parameter sets is taken as the set of network parameters corresponding to the frame to be encoded.

7. The method according to any one of claims 1 to 6, characterized in that, A preset network parameter set includes multiple preset network parameter groups; The encoding network includes multiple sub-networks, and the multiple sub-networks correspond to the multiple preset network parameter groups respectively; The preset network parameter groups of any two preset network parameter sets in the M preset network parameter sets are different.

8. The method according to any one of claims 1 to 7, characterized in that, The bitstream of the frame to be encoded includes a set identifier of the network parameter set corresponding to the frame to be encoded.

9. A decoding method, characterized in that, The method includes: Obtain the bitstream of the frame to be decoded, wherein the frame to be decoded is a P-frame; From M preset network parameter sets, determine the network parameter set corresponding to the frame to be decoded, wherein the M preset network parameter sets correspond to different compression performance information, and M is an integer greater than 1; The decoding network decodes the bitstream of the frame to be decoded according to the set of network parameters corresponding to the frame to be decoded, so as to obtain the reconstructed image of the frame to be decoded; In the decoding process, the decoding network uses different sets of network parameters to decode multiple adjacent P-frames, so that the compression performance of the multiple adjacent P-frames varies.

10. The method according to claim 9, characterized in that, The M equals 2, and the M preset network parameter sets include a first preset network parameter set and a second preset network parameter set; The step of determining the network parameter set corresponding to the frame to be decoded from M preset network parameter sets includes: When the frame identifier of the frame to be decoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded; When the frame identifier of the frame to be decoded is an even number, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded.

11. The method according to claim 10, characterized in that, The compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set. or, The compression performance information corresponding to the second preset network parameter set is higher than that corresponding to the first preset network parameter set.

12. The method according to claim 9, characterized in that, The step of determining the network parameter set corresponding to the frame to be decoded from M preset network parameter sets includes: Parse the set identifier from the bitstream of the frame to be decoded; The preset network parameter set that matches the set identifier among the M preset network parameter sets is determined as the network parameter set corresponding to the frame to be decoded.

13. The method according to claim 9, characterized in that, The step of determining the network parameter set corresponding to the frame to be decoded from M preset network parameter sets includes: When the network parameter sets corresponding to the previous N frames of the frame to be decoded are the same, according to the preset rules, one preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be decoded is selected as the network parameter set corresponding to the frame to be decoded. Otherwise, the set of network parameters corresponding to the previous frame of the frame to be decoded is determined as the set of network parameters corresponding to the frame to be decoded, where N is an integer greater than 1.

14. An encoding system, characterized in that, The encoding system includes: a selector and an encoding network, wherein: The selector is used to determine the network parameter set corresponding to the frame to be encoded from M preset network parameter sets, wherein the frame to be encoded is a P-frame, and the M preset network parameter sets correspond to different compression performance information, where M is an integer greater than 1; The encoding network is used to encode the frame to be encoded according to the set of network parameters corresponding to the frame to be encoded, so as to obtain the bitstream of the frame to be encoded; In the encoding process, the encoding network uses different sets of network parameters to encode multiple adjacent P-frames, so that the compression performance of the multiple adjacent P-frames varies.

15. The system according to claim 14, characterized in that, The M equals 2, and the M preset network parameter sets include a first preset network parameter set and a second preset network parameter set; The step of determining the network parameter set corresponding to the frame to be encoded from M preset network parameter sets includes: When the frame identifier of the frame to be encoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded; When the frame identifier of the frame to be encoded is an even number, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be encoded.

16. The system according to claim 15, characterized in that, The compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set. or, The compression performance information corresponding to the second preset network parameter set is higher than that corresponding to the first preset network parameter set.

17. The system according to claim 14, characterized in that, The step of determining the network parameter set corresponding to the frame to be encoded from M preset network parameter sets includes: When the network parameter sets corresponding to the previous N frames of the frame to be encoded are the same, according to the preset rules, one preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be encoded is selected as the network parameter set corresponding to the frame to be encoded. Otherwise, the set of network parameters corresponding to the previous frame of the frame to be encoded is determined as the set of network parameters corresponding to the frame to be encoded, where N is an integer greater than 1.

18. The system according to claim 14, characterized in that, The step of determining the network parameter set corresponding to the frame to be encoded from M preset network parameter sets includes: Determine the compression performance information corresponding to each of the M preset network parameter sets; Based on the compression performance information corresponding to the M preset network parameter sets, the network parameter set corresponding to the frame to be encoded is determined from the M preset network parameter sets.

19. The system according to claim 18, characterized in that, The step of determining the network parameter set corresponding to the frame to be encoded from the M preset network parameter sets based on the compression performance information corresponding to each of the M preset network parameter sets includes: When the compression performance information corresponding to the first K frames of the frame to be encoded is higher than the threshold, a preset network parameter set whose compression performance information is lower than the threshold from the M preset network parameter sets is taken as the network parameter set corresponding to the frame to be encoded; if the compression performance information corresponding to the first K frames of the frame to be encoded is lower than the threshold, a preset network parameter set whose compression performance information is higher than the threshold from the M preset network parameter sets is taken as the network parameter set corresponding to the frame to be encoded, where K is a positive integer; Otherwise, if the compression performance information corresponding to the previous frame of the frame to be encoded is lower than the threshold, then the set of preset network parameters whose compression performance information is lower than the threshold from the M preset network parameter sets is taken as the set of network parameters corresponding to the frame to be encoded; if the compression performance information corresponding to the previous frame of the frame to be encoded is higher than the threshold, then the set of preset network parameters whose compression performance information is higher than the threshold from the M preset network parameter sets is taken as the set of network parameters corresponding to the frame to be encoded.

20. A decoding system, characterized in that, The decoding system includes: a selector and a decoding network, wherein: The selector is used to determine the network parameter set corresponding to the frame to be decoded from M preset network parameter sets, wherein the frame to be decoded is a P-frame, and the M preset network parameter sets correspond to different compression performance information, where M is an integer greater than 1; The decoding network is used to decode the bitstream of the frame to be decoded according to the network parameter set corresponding to the frame to be decoded, so as to obtain the reconstructed image of the frame to be decoded. In the decoding process, the decoding network uses different sets of network parameters to decode multiple adjacent P-frames, so that the compression performance of the multiple adjacent P-frames varies.

21. The system according to claim 20, characterized in that, The M equals 2, and the M preset network parameter sets include a first preset network parameter set and a second preset network parameter set; The step of determining the network parameter set corresponding to the frame to be decoded from M preset network parameter sets includes: When the frame identifier of the frame to be decoded is odd, the first preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded; When the frame identifier of the frame to be decoded is an even number, the second preset network parameter set is determined as the network parameter set corresponding to the frame to be decoded.

22. The system according to claim 21, characterized in that, The compression performance information corresponding to the first preset network parameter set is higher than the compression performance information corresponding to the second preset network parameter set. or, The compression performance information corresponding to the second preset network parameter set is higher than that corresponding to the first preset network parameter set.

23. The system according to claim 20, characterized in that, The step of determining the network parameter set corresponding to the frame to be decoded from M preset network parameter sets includes: Parse the set identifier from the bitstream of the frame to be decoded; The preset network parameter set that matches the set identifier among the M preset network parameter sets is determined as the network parameter set corresponding to the frame to be decoded.

24. The system according to claim 20, characterized in that, The step of determining the network parameter set corresponding to the frame to be decoded from M preset network parameter sets includes: When the network parameter sets corresponding to the previous N frames of the frame to be decoded are the same, according to the preset rules, one preset network parameter set other than the network parameter set corresponding to the previous frame of the frame to be decoded is selected as the network parameter set corresponding to the frame to be decoded. Otherwise, the set of network parameters corresponding to the previous frame of the frame to be decoded is determined as the set of network parameters corresponding to the frame to be decoded, where N is an integer greater than 1.

25. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the encoding method according to any one of claims 1 to 8.

26. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the decoding method according to any one of claims 9 to 13.

27. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of the electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the method of any one of claims 1 to 13.

28. A computer storage medium, characterized in that, The computer storage medium stores a computer program that, when run on the computer, causes the computer to perform the method as described in any one of claims 1 to 13.

29. A computer program product, characterized in that, The computer program product includes a software program that, when executed by a computer, causes the steps of the method according to any one of claims 1 to 13 to be performed.

Images