Image decoding and encoding method, apparatus, device, and storage medium

Abstract

The application belongs to the technical field of image processing, and discloses an image decoding and encoding method, device, equipment and storage medium. The application obtains a plurality of extended residual groups by grouping the extended residual data extracted from the image code stream; performs residual recovery on the plurality of extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group; performs enlarged spatial resolution processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data; and performs image reconstruction according to the reconstruction feature data to obtain a reconstruction image block. Since the obtained extended residual data is residual data subjected to reduced spatial resolution processing, the residual recovery processing can be performed on the whole group in a low resolution, thereby improving the overall residual recovery calculation efficiency and reducing the time complexity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an image decoding and encoding method, device, equipment and storage medium. BACKGROUND

[0002] In a deep learning-based image compression scheme, the decoded feature points are used as prior information to perform mean prediction on the feature points being decoded, so as to reduce the spatial redundancy of the image. The mainstream scheme generally adopts serial or wave-front coding and decoding, and the serial degree is increased with the features, and the overall execution time complexity is high.

[0003] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide an image decoding and encoding method, device, equipment and storage medium, which aims to solve the technical problem of high time complexity in the mean prediction of the image coding and decoding process in the prior art.

[0005] To achieve the above purpose, the present application provides an image decoding method, which comprises the following steps:

[0006] Grouping the extracted extended residual data from the image code stream to obtain a plurality of extended residual groups;

[0007] Residual recovery is performed on the plurality of extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group;

[0008] The image reconstruction features corresponding to each extended residual group are subjected to enlarged spatial resolution processing to obtain reconstruction feature data;

[0009] Image reconstruction is performed according to the reconstruction feature data to obtain a reconstructed image block.

[0010] In a possible implementation of the present application, the grouping of the extracted extended residual data from the image code stream to obtain a plurality of extended residual groups comprises:

[0011] Extended residual data is extracted from the image code stream;

[0012] The extended residual data is grouped based on the feature channels corresponding to the extended residual data to obtain a plurality of extended residual groups.

[0013] In a possible implementation of the present application, the grouping of the extracted extended residual data from the image code stream to obtain a plurality of extended residual groups comprises:

[0014] image residual data extracted from the image code stream;

[0015] performing down-spatial-resolution processing on the image residual data to obtain extended residual data, the up-spatial-resolution processing being inverse of the down-spatial-resolution processing;

[0016] grouping the extended residual data to obtain a plurality of extended residual groups.

[0017] In a possible implementation of the present application, the down-spatial-resolution processing on the image residual data extracted from the image code stream to obtain extended residual data comprises:

[0018] grouping data according to the feature channels corresponding to the image residual data to obtain at least one image residual group;

[0019] performing down-spatial-resolution processing on the data in the image residual group to obtain extended residual data.

[0020] In a possible implementation of the present application, the down-spatial-resolution processing on the image residual data to obtain extended residual data comprises:

[0021] reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain extended residual data.

[0022] In a possible implementation of the present application, the reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain extended residual data comprises:

[0023] reducing the spatial size corresponding to the image residual data according to the spatial information corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain extended residual data;

[0024] or,

[0025] reducing the spatial size corresponding to the image residual data according to the frequency information corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain extended residual data;

[0026] or,

[0027] reducing the spatial size corresponding to the image residual data according to a preset convolution layer and / or increasing the number of feature channels corresponding to the image residual data to obtain extended residual data.

[0028] In a possible implementation of the present application, the step of performing residual recovery on the plurality of extended residual groups based on the residual recovery sequence to obtain image reconstruction features corresponding to each extended residual group comprises:

[0029] constructing a residual recovery sequence according to the plurality of extended residual groups;

[0030] performing residual recovery on the plurality of extended residual groups based on the residual recovery sequence to obtain image reconstruction features corresponding to each extended residual group.

[0031] In a possible implementation of the present application, the step of performing residual recovery on the plurality of extended residual groups based on the residual recovery sequence to obtain image reconstruction features corresponding to each extended residual group comprises:

[0032] traversing the residual recovery sequence to obtain a current extended residual group;

[0033] obtaining auxiliary information output by an auxiliary encoding network;

[0034] constructing prior information according to the auxiliary information;

[0035] performing residual recovery on the current extended residual group based on the prior information to obtain an image reconstruction feature corresponding to the current extended residual group;

[0036] obtaining image reconstruction features corresponding to each extended residual group at the end of the traversal.

[0037] In a possible implementation of the present application, the step of constructing prior information according to the auxiliary information comprises:

[0038] performing down-spatial-resolution processing on the auxiliary information to obtain extended auxiliary information;

[0039] detecting whether the current extended residual group is the first element in the residual recovery sequence;

[0040] if the current extended residual group is the first element, constructing prior information according to the extended auxiliary information;

[0041] if the current extended residual group is not the first element, concatenating the extended auxiliary information and a convolution processing result corresponding to an image reconstruction feature of a recovered extended residual group to obtain concatenated auxiliary information, and constructing prior information according to the concatenated auxiliary information.

[0042] In a possible implementation of the present application, the step of concatenating the extended auxiliary information and a convolution processing result corresponding to an image reconstruction feature of a recovered extended residual group to obtain concatenated auxiliary information comprises:

[0043] obtaining an image reconstruction feature corresponding to the recovered extended residual group;

[0044] performing feature enhancement on the image reconstruction feature to obtain an enhanced reconstruction feature;

[0045] splicing the auxiliary information and a convolution processing result corresponding to the enhanced reconstruction feature to obtain spliced auxiliary information.

[0046] In a possible implementation of the present application, the performing feature enhancement on the image reconstruction feature to obtain an enhanced reconstruction feature comprises:

[0047] obtaining a prediction mean value, auxiliary information, image residual data and / or residual data variance corresponding to the recovered extended residual group;

[0048] performing feature enhancement on the image reconstruction feature according to the prediction mean value, auxiliary information, image residual data and / or residual data variance corresponding to the recovered extended residual group to obtain an enhanced reconstruction feature.

[0049] In a possible implementation of the present application, the performing upscaling spatial resolution processing on the image reconstruction feature corresponding to each extended residual group to obtain reconstruction feature data comprises:

[0050] performing feature enhancement on the image reconstruction feature corresponding to each extended residual group to obtain enhanced reconstruction feature corresponding to each extended residual group;

[0051] performing upscaling spatial resolution processing on the enhanced reconstruction feature corresponding to each extended residual group to obtain reconstruction feature data.

[0052] In addition, to achieve the above object, the present application further provides an image coding method, which comprises:

[0053] performing downscaling spatial resolution processing on image features corresponding to a to-be-coded image to obtain extended image features;

[0054] grouping the extended image features to obtain a plurality of extended feature groups;

[0055] performing residual calculation on the plurality of extended feature groups respectively to obtain image residual data corresponding to each extended feature group;

[0056] generating an image code stream according to the image residual data and sending the image code stream to an image decoding end.

[0057] In a possible implementation of the present application, the performing downscaling spatial resolution processing on image features corresponding to a to-be-coded image to obtain extended image features comprises:

[0058] obtaining image features corresponding to a to-be-coded image;

[0059] group the image features according to feature channels corresponding to the image features, to obtain at least one image feature group;

[0060] perform down-spatial resolution processing on data in the image feature group, to obtain extended image features.

[0061] In a possible implementation of the present application, the down-spatial resolution processing on the image features corresponding to the image to be encoded to obtain extended image features comprises:

[0062] reducing spatial dimensions of the image features corresponding to the image to be encoded, and / or increasing the number of feature channels corresponding to the image features, to obtain the extended image features.

[0063] In a possible implementation of the present application, the reducing spatial dimensions of the image features corresponding to the image to be encoded, and / or increasing the number of feature channels corresponding to the image features, to obtain the extended image features comprises:

[0064] reducing spatial dimensions of the image features corresponding to the image to be encoded according to spatial information or frequency domain information of the image features, and / or increasing the number of feature channels corresponding to the image features, to obtain the extended residual data;

[0065] or,

[0066] reducing spatial dimensions of the image features corresponding to the image to be encoded according to a preset convolution layer, and / or increasing the number of feature channels corresponding to the image features, to obtain the extended residual data.

[0067] In a possible implementation of the present application, the performing residual calculation on the plurality of extended feature groups respectively to obtain image residual data corresponding to each extended feature group comprises:

[0068] constructing a residual calculation sequence according to the plurality of extended feature groups;

[0069] performing residual calculation on the plurality of extended feature groups respectively based on the residual calculation sequence, to obtain image residual data corresponding to each extended feature group.

[0070] In a possible implementation of the present application, the performing residual calculation on the plurality of extended feature groups respectively based on the residual calculation sequence, to obtain image residual data corresponding to each extended feature group comprises:

[0071] traversing the residual calculation sequence to obtain a current extended feature group;

[0072] obtaining auxiliary information output by an auxiliary encoding network;

[0073] constructing prior information according to the auxiliary information;

[0074] perform residual calculation on the current extended feature group based on the prior information to obtain image residual data corresponding to the current extended feature group;

[0075] At the end of the traversal, image residual data corresponding to each extended feature group is obtained.

[0076] In a possible implementation of the present application, the grouping of the extended image features to obtain a plurality of extended feature groups comprises:

[0077] The extended image features are grouped based on feature channels corresponding to the extended feature data to obtain a plurality of extended residual groups.

[0078] In a possible implementation of the present application, the generation of an image code stream according to the image residual data and the sending of the image code stream to an image decoding end comprises:

[0079] The image residual data corresponding to the to-be-encoded image is obtained by performing upscaling spatial resolution processing on the image reconstruction features corresponding to each extended feature group.

[0080] The image code stream is generated according to the image residual data corresponding to the to-be-encoded image, and the image code stream is sent to an image decoding end.

[0081] In addition, to achieve the above-mentioned purpose, the present application further provides an image decoding device, which comprises:

[0082] The code stream decoding module is configured to group the extended residual data extracted from the image code stream to obtain a plurality of extended residual groups.

[0083] The residual recovery module is configured to perform residual recovery on the plurality of extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group.

[0084] The data combination module is configured to perform upscaling spatial resolution processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data.

[0085] The image reconstruction module is configured to perform image reconstruction according to the reconstruction feature data to obtain a reconstructed image block.

[0086] In addition, to achieve the above-mentioned purpose, the present application further provides an image encoding device, which comprises:

[0087] The feature extraction module is configured to perform downscaling spatial resolution processing on image features corresponding to a to-be-encoded image to obtain extended image features.

[0088] a data grouping module, configured to group the extended image features to obtain a plurality of extended feature groups;

[0089] a residual calculation module, configured to perform residual calculation on the plurality of extended feature groups respectively to obtain image residual data corresponding to each extended feature group;

[0090] a code stream generation module, configured to generate an image code stream according to the image residual data and send the image code stream to an image decoding end.

[0091] In addition, to achieve the above object, the present application further provides a decoding device, which comprises a processor, a memory, and an image decoding program stored in the memory and executable on the processor, and the image decoding program implements the steps of the image decoding method when executed by the processor.

[0092] In addition, to achieve the above object, the present application further provides an encoding device, which comprises a processor, a memory, and an image decoding program and / or an image encoding program stored in the memory and executable on the processor, the image decoding program implements the steps of the image decoding method when executed by the processor, and the image encoding program implements the steps of the image encoding method when executed by the processor.

[0093] In addition, to achieve the above object, the present application further provides a storage medium, which stores an image decoding program and / or an image encoding program, the image decoding program implements the steps of the image decoding method when executed, and the image encoding program implements the steps of the image encoding method when executed.

[0094] The present application groups the extracted extended residual data from the image code stream to obtain a plurality of extended residual groups, performs residual recovery on the plurality of extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group, performs upsampling spatial resolution processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data, and performs image reconstruction according to the reconstruction feature data to obtain a reconstruction image block. Since the obtained extended residual data is residual data subjected to downsampling spatial resolution processing, the residual recovery processing can be performed on the whole group in a low resolution, thereby improving the overall residual recovery calculation efficiency and reducing the time complexity. BRIEF DESCRIPTION OF DRAWINGS

[0095] Figure 1 is a structural schematic diagram of an electronic device of a hardware running environment related to the embodiment scheme of the present application;

[0096] Figure 2 is a flowchart of the first embodiment of the image encoding method of the present application;

[0097] Figure 3 Figure 1 is a schematic diagram of an image compression framework according to an embodiment of the present application;

[0098] Figure 4 Figure 2 is a schematic diagram of an image encoding method according to a second embodiment of the present application;

[0099] Figure 5 Figure 3 is a schematic diagram of a spatial resolution processing flow according to an embodiment of the present application

[0100] Figure 6 Figure 4 is a schematic diagram of an image encoding method according to a second embodiment of the present application;

[0101] Figure 7 Figure 5 is a schematic diagram of an image decoding packet execution flow according to an embodiment of the present application;

[0102] Figure 8 Figure 6 is a schematic diagram of a secondary packet execution flow according to an embodiment of the present application;

[0103] Figure 9 Figure 7 is a schematic diagram of a feature enhancement packet execution flow according to an embodiment of the present application;

[0104] Figure 10 Figure 8 is a schematic diagram of an image encoding method according to a first embodiment of the present application;

[0105] Figure 11 Figure 9 is a schematic diagram of an image encoding method according to a second embodiment of the present application;

[0106] Figure 12 Figure 10 is a structural block diagram of an image decoding apparatus according to a first embodiment of the present application;

[0107] Figure 13 Figure 11 is a structural block diagram of an image encoding apparatus according to a first embodiment of the present application.

[0108] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0109] It should be understood that the specific embodiments described herein merely exemplify the application and do not limit the application.

[0110] Reference Figure 1 , Figure 1 Figure 12 is a structural schematic diagram of a decoding device or an encoding device according to an embodiment of the present application.

[0111] As Figure 1As shown, the electronic device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display, an input unit such as a keyboard, and can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (WI-FI) interface). The memory 1005 can be a high-speed random access memory (RAM), and can also be a stable non-volatile memory (NVM), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0112] Those skilled in the art can understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0113] As Figure 1 As shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and an image decoding program and / or an image encoding program.

[0114] In Figure 1 As shown in the electronic device, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present application can be arranged in a decoding device or an encoding device, and the electronic device calls the image decoding program or the image encoding program stored in the memory 1005 through the processor 1001, and executes the image decoding method or the image encoding method provided by the embodiments of the present application.

[0115] The embodiment of the present application provides an image decoding method, which refers to Figure 2 , Figure 2 The flowchart of a first embodiment of the image decoding method of the present application.

[0116] In this embodiment, the image decoding method includes the following steps:

[0117] Step S10: Grouping the extracted extended residual data from the image code stream to obtain a plurality of extended residual groups.

[0118] It should be noted that the execution subject of the present embodiment can be a decoding device in the process of encoding image data, which can be a personal computer, a server, or other electronic devices, and of course, other devices that can achieve the same or similar functions, which are not limited in the present embodiment. In the present embodiment and the following embodiments, the decoding device is taken as an example to describe the image decoding method of the present application.

[0119] In the process of image encoding, the encoding device generally decodes the encoded image code stream after encoding, and determines whether the parameters used in the encoding process need to be adjusted according to the image quality of the decoded image. Therefore, the execution subject of the present embodiment can also be the encoding device.

[0120] It should be noted that the image code stream can be a code stream generated by the encoding device after encoding the image data to be compressed and encoded. When generating the image code stream, the encoding device reduces the spatial resolution of the image features to reduce the time complexity of the prediction mean in the encoding process. Finally, the encoding device directly encodes the generated extended residual data into the image code stream. At this time, the decoding device can directly extract the extended residual data from the image code stream. At this time, the decoding device can divide the extended residual data into multiple extended residual groups, and then recover the residual one by one, thereby reducing the time complexity of the mean prediction in the image decoding process.

[0121] The technical terms involved in the process of encoding or decoding images include: JPEG (Joint Photographic Experts Group), JPEG-AI (Joint Photographic Experts Group Artificial Intelligence), entropy encoding (Entropy Encoding), neural network (Neural Network, NN), convolutional neural network (Convolutional Neural Network, CNN), feature (feature), rate-distortion principle (Rate-Distortion Optimized), etc., which are described herein.

[0122] Wherein, JPEG (Joint Photographic Experts Group) is a standard for continuous-tone still image compression, and the file suffix is.jpg or.jpeg. It is the most commonly used image file format. Its main joint coding method is prediction coding (DPCM), discrete cosine transform (DCT) and entropy coding to remove redundant image and color data. It belongs to lossy compression format, which can compress the image into a small storage space, and to a certain extent, it will cause damage to the image data. Especially using too high compression ratio will reduce the quality of the image recovered after final decompression. If high-quality images are pursued, it is not appropriate to use too high compression ratio.

[0123] The scope of JPEG AI is to create a learning-based image coding standard that provides a single-stream, compact, compressed-domain representation that is both human visualizable and significantly improves compression efficiency over commonly used image coding standards for equivalent subjective quality, as well as effective performance for image processing and computer vision tasks. JPEG AI targets a wide range of applications, such as cloud storage, visual surveillance, autonomous cars and devices, image acquisition, storage and management, real-time monitoring of visual data, and media distribution. The goal is to design an encoding solution that significantly improves the compression efficiency of commonly used coding standards with the same subjective quality and provides effective compressed-domain processing for machine learning-based image processing and computer vision tasks. Other key requirements include hardware / software implementation-friendly encoding and decoding, support for 8-bit and 10-bit depth, efficient encoding of images using text and graphics, and progressive decoding.

[0124] Entropy coding is a coding process that does not lose any information according to the entropy principle. Information entropy is the average amount of information of the source (a measure of uncertainty). Common entropy coding includes Shannon coding, Huffman coding and arithmetic coding.

[0125] The neural network in the present application refers to an artificial neural network, rather than a biological neural network. The neural network is an operation model composed of a large number of nodes (or neurons) connected with each other. In the artificial neural network, the neuron processing unit can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in the network are divided into three categories: input units, output units and hidden units. The input units accept signals and data from the external world; the output units output the processing results of the system; the hidden units are the units between the input and output units and cannot be observed from the outside of the system. The connection weights between neurons reflect the connection strength between units, and the information representation and processing are embodied in the connection relationship of the network processing units. The artificial neural network is a non-programmed, brain-like information processing method, the essence of which is to obtain a parallel distributed information processing function through the transformation and dynamics of the network, and to simulate the information processing function of the human brain neural system at different levels and levels. At present, in the field of video processing, commonly used neural networks include convolutional neural network (CNN), recurrent neural network (RNN), fully connected network, etc.

[0126] The convolutional neural network is a kind of feedforward neural network, which is one of the most representative network structures in deep learning technology. The artificial neurons of the convolutional neural network can respond to a part of the surrounding units in the coverage range, and have excellent performance for large image processing. Generally, the basic structure of CNN includes two layers, one of which is a feature extraction layer (also called a convolution layer), and the input of each neuron is connected with the local receptive field of the previous layer, and the local features are extracted. Once the local features are extracted, the positional relationship between them and other features is also determined; the second is a feature mapping layer (also called an activation layer), each calculation layer of the network is composed of multiple feature mappings, and each feature mapping is a plane, and all the neurons on the plane have equal weights. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since the neurons on a mapping plane share weights, the number of free parameters of the network is reduced. One of the advantages of CNN compared with traditional image processing algorithms is that it avoids the complex pre-processing process (extracting artificial features, etc.) of the image, and can directly input the original image for end-to-end learning. One of the advantages of CNN compared with traditional neural networks is that traditional neural networks all use full connection, that is, all the neurons from the input layer to the hidden layer are connected, which will result in a large number of parameters, making the network training time-consuming and even difficult to train, while CNN avoids this difficulty through local connection and weight sharing.

[0127] The feature involved in the present application is a three-dimensional feature matrix of CxWxH (as shown in Figure 3 ​Figure 3 The matrix structure of the embodiment is shown in the following table (the table is a schematic diagram of the matrix structure of the embodiment). C represents the number of channels, H represents the feature height, and W represents the feature width. The feature matrix can be the input of the neural network or the output of the neural network.

[0128] The indicators for evaluating the coding efficiency include code rate, PSNR, MS-SSIM, VMAF FSIM, PSNRHVS, and various other indicators, and of course, more indicators can be included, which are not limited herein. The smaller the bit stream is, the greater the compression rate is; the greater the PSNR is, the better the image coding efficiency is. In the mode selection, the discrimination formula is essentially a comprehensive evaluation of the two. The cost corresponding to the mode is J(mode) = D + l * R. D represents Distortion, which is usually measured by using the SSE indicator, and SSE is the sum of squares of differences between the reconstructed block and the source image. l is the Lagrange multiplier. R is the actual number of bits required for image block coding in the mode, including the total number of bits required for coding mode information, residual error, and the like.

[0129] In a possible implementation of the embodiment, the obtained extended residual error data is grouped according to the feature channels corresponding to the extended residual error data. At this time, the step S10 of the embodiment can include:

[0130] extracting the extended residual error data from the image code stream;

[0131] grouping the extended residual error data according to the feature channels corresponding to the extended residual error data to obtain a plurality of extended residual error groups.

[0132] It should be noted that grouping the extended residual error data according to the feature channels corresponding to the extended residual error data to obtain a plurality of extended residual error groups can be to divide the extended residual error data into a plurality of groups according to the corresponding feature channels, for example, assuming that the total number of feature channels corresponding to the extended residual error data is 20, at this time, the extended residual error data corresponding to the feature channels 1-10 can be divided into a group, and the extended residual error data corresponding to the feature channels 11-20 can be divided into a group. The number of groups of uniform division can be set by the administrator of the decoding device in advance, and the embodiment does not limit this.

[0133] Of course, the grouping can also be non-uniform, at this time, grouping the extended residual error data according to the feature channels corresponding to the extended residual error data to obtain a plurality of extended residual error groups can also be to divide the extended residual error data into a plurality of groups according to the corresponding feature channels based on a preset grouping rule. The preset grouping rule can be set by the administrator of the decoding device according to actual needs in advance, for example, the preset grouping rule is set to divide the extended residual error data of the first m / n (n is the total number of feature channels, and m is a preset value, the value range is [1, n)) into a group, and the remaining extended residual error data into another group.

[0134] In practical applications, when grouping extended residual data based on the feature channels corresponding to the extended residual data to obtain multiple extended residual groups, it is also possible to divide the extended residual data corresponding to one feature channel into one group. For example, assuming that the total number of feature channels corresponding to the extended residual data is 20, the extended residual data can be divided into 20 groups according to the different feature channels.

[0135] Step S20: Perform residual recovery on the multiple extended residual groups respectively to obtain the image reconstruction features corresponding to each extended residual group.

[0136] It should be noted that residual recovery of the extended residual group to obtain the image reconstruction features corresponding to the extended residual group can be achieved by predicting the mean of the extended residual group, and then summing the mean values ​​predicted from the residual data domains in the extended residual group to obtain the image reconstruction features corresponding to the extended residual group.

[0137] Step S30: Perform spatial resolution amplification processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data.

[0138] It should be noted that since the extended residual data has undergone spatial resolution reduction processing, the spatial size and number of channels corresponding to each data are different from the spatial size and number of channels of the image features obtained by the encoding device from the original image feature extraction. In order to ensure the smooth execution of image reconstruction, the spatial resolution of the image reconstruction features corresponding to each extended residual group can be increased to restore the spatial size and number of channels of the image reconstruction features to be consistent with the image features obtained from the original image feature extraction.

[0139] Among them, the process of increasing spatial resolution can be the reverse process of decreasing spatial resolution in the encoding device.

[0140] Step S40: Reconstruct the image based on the reconstruction feature data to obtain a reconstructed image block.

[0141] It should be noted that after obtaining reconstructed feature data that has the same spatial size and number of channels as the image features corresponding to the original image, image reconstruction can be performed based on the reconstructed feature data to obtain reconstructed image blocks.

[0142] In this process, image reconstruction based on reconstruction feature data to obtain reconstructed image patches can be achieved by using a pre-built synthetic transformation network to perform synthetic transformation processing on the reconstruction feature data, thereby obtaining reconstructed image patches. The synthetic transformation network can be a network built based on deep learning or a neural network.

[0143] To facilitate understanding, we will now combine...Figure 3 The following will be explained to illustrate but not to limit the present application. Figure 3 The overall framework of image compression is shown in the following figure. Figure 3 As shown in the figure, at the encoding device end: x is the input image, and a latent representation y is generated through a main encoder (Analysis Transform Net). Direct encoding of y requires a large amount of code rate, so a context (Context Model Net) and a set of auxiliary networks (Hyper Encoder / Deocder Net) are introduced to make a prediction to obtain a prediction result μ, and a residual resi=y-μ is obtained, and the distribution parameters σ of the residual are obtained through the auxiliary networks (Hyper Encoder Net and Hyper Decoder Net). The distribution parameters can help the entropy encoding to encode data with less code rate. Therefore, at the encoding and decoding end, the prediction mean μ and the distribution parameters σ of each y need to be obtained to correctly perform entropy encoding and decoding. The resi and the distribution parameters σ are scaled through a G-unit component to control the loss of quantization. The resi is rounded to obtain the quantized residual, denoted as Finally, lossless entropy encoding (Lossless encoder) is performed.

[0144] At the decoding device end: the image code stream is entropy decoded to obtain The invG-unit component scales , wherein the scaling multiples of the G-unit and the invG-unit modules are opposite, and after scaling, the prediction value μ is combined to obtain The obtained is input into a synthesis transform network (Synthesis Transform Net) at the decoding end to obtain a reconstructed image block.

[0145] The present embodiment obtains a plurality of extended residual groups by grouping the extracted extended residual data from the image code stream; obtains image reconstruction features corresponding to each extended residual group by respectively performing residual recovery on the plurality of extended residual groups; performs upsampling spatial resolution processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data; and performs image reconstruction according to the reconstruction feature data to obtain a reconstructed image block. Since the obtained extended residual data is residual data that has been subjected to downsampling spatial resolution processing, the residual recovery processing can be performed on the whole group at a low resolution, thereby improving the overall residual recovery calculation efficiency and reducing the time complexity.

[0146] Reference Figure 4 , Figure 4 The following figure is a flowchart of a second embodiment of an image decoding method of the present application.

[0147] Based on the above first embodiment, the step S10 of the image decoding method of the present embodiment comprises:

[0148] Step S101: extracting image residual data from the image code stream.

[0149] It should be noted that the encoding device can perform upsampling spatial resolution processing on the generated extended residual data to restore the image residual data with the same image features as the original image in terms of spatial size and the number of channels, and then encode the image residual data into the image code stream. In this case, the decoding device can only extract the image residual data from the image code stream when decoding the image code stream.

[0150] Step S102: downsampling spatial resolution processing is performed on the image residual data to obtain extended residual data.

[0151] It can be understood that, in order to facilitate subsequent grouping processing and reduce time complexity, after obtaining the image residual data, downsampling spatial resolution processing can be performed on the image residual data to obtain extended residual data. In order to ensure the correctness of decoding, the downsampling spatial resolution processing performed by the encoding device and the decoding device needs to be consistent, and the upsampling spatial resolution processing is the inverse process of the downsampling spatial resolution processing.

[0152] After downsampling spatial resolution processing is performed on the image residual data, the spatial size of the image residual data will be smaller, and therefore, a smaller convolution kernel can be used when processing the image residual data. For example, if a 5x5 convolution kernel is used on the original image residual data, a 3x3 convolution kernel is equivalent to a 5x5 convolution kernel in the receptive field of the convolution kernel after downsampling spatial resolution processing.

[0153] In actual use, the step of downsampling spatial resolution processing on the image residual data to obtain extended residual data in the embodiment can include:

[0154] The spatial size corresponding to the image residual data is reduced, and / or the number of feature channels corresponding to the image residual data is increased to obtain extended residual data.

[0155] It should be noted that the reduction amplitude of the spatial size corresponding to the image residual data and the increase amplitude of the number of feature channels corresponding to the image residual data can be set by the manager of the decoding device in advance, and the embodiment does not limit this.

[0156] In actual execution, only the reduction amplitude of the spatial size can be set, or only the increase amplitude of the number of feature channels can be set. The decoding device can adjust the spatial size or the number of feature channels adaptively, or the decoding device can be set to execute the reduction amplitude of the spatial size and the increase amplitude of the number of feature channels without using device adaptation.

[0157] For example, assuming that the image feature corresponding to the image residual data is y∈R {H,W,C} , where H is the height of the image feature, W is the width of the image feature, and C is the number of feature channels corresponding to the image feature. At this time, H and W can be reduced to half of the original, and then in order to ensure that the data volume remains unchanged, the number of feature channels will be 4 times the original. At this time, the image feature corresponding to the expanded residual data can be represented as y∈R {H / 2,W / 2,4C} .

[0158] In a specific implementation, the spatial resolution reduction processing can be performed based on the spatial domain information or the frequency information corresponding to the image residual data, and the spatial resolution reduction processing can also be performed through a preset convolution layer. At this time, the step of reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain expanded residual data can include:

[0159] reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data according to the spatial domain information corresponding to the image residual data to obtain expanded residual data;

[0160] or,

[0161] reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data according to the frequency domain information corresponding to the image residual data to obtain expanded residual data;

[0162] or,

[0163] reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data according to a preset convolution layer to obtain expanded residual data.

[0164] For the convenience of understanding, the following will be described with reference to Figure 5 , but the scheme is not limited thereto. Figure 5 is a spatial resolution processing flowchart of the embodiment, Figure 5 a and Figure 5 b are spatial resolution processing according to the spatial domain information corresponding to the image residual data, as shown in Figure 5 a, SpaceShuffle, the data is resampled and arranged, and the result is that all points with the same spatial domain position are converged and arranged. UnspaceShuffle is the inverse process, and the downsampling method is lossless. That is, after the data passes through SpaceShuffle and UnspaceShuffle, it is the same as the source data.

[0165] For example,Figure 5 b. Pixshuffle, which has a similar sampling manner as a, but the difference is that the data is interleaved in the channel dimension. The data of the previous channel is divided into four channels according to the spatial domain, and then arranged in sequence. Figure 5 a. Pixshuffle, which has a similar sampling manner as a, but the difference is that the data is interleaved in the channel dimension. The data of the previous channel is divided into four channels according to the spatial domain, and then arranged in sequence.

[0166] Figure 5 c. Spatial resolution processing according to the frequency domain information corresponding to the image residual data, as shown in c. Figure 5 c. Wavelet transform is a two-dimensional wavelet transform, which can divide the data in the frequency domain and output four frequency domain subbands with a dimension of [H / 2, W / 2, C]. Each subband represents a different frequency domain feature. Inv Wavelet transform is the inverse process of Wavelet transform, which combines the frequency domain subbands into the original data. This process is lossless.

[0167] Figure 5 d. Spatial resolution processing according to a preset convolution layer, as shown in d. Figure 5 d. Convlution is a convolution transform, which directly inputs the convolution layer for processing and obtains an output of [H / 2, W / 2, 4*C], achieving the purpose of reducing the spatial resolution. The corresponding inverse process also uses a convolution layer for processing, restoring to the original image size. However, this method is lossy, and the data after processing has differences with the source data.

[0168] In a possible implementation manner of the embodiment, before the image residual data is processed to reduce the spatial resolution, the image residual data can be grouped first, and then processed to reduce the spatial resolution. At this time, the step S102 of the embodiment includes:

[0169] grouping the data according to the feature channels corresponding to the image residual data to obtain at least one image residual group;

[0170] processing the data in the image residual group to reduce the spatial resolution to obtain extended residual data.

[0171] It should be noted that the same or similar method as grouping the extended residual data can be used when grouping the data according to the feature channels corresponding to the image residual data. The data in the image residual group can be processed to reduce the spatial resolution, and then the data after the spatial resolution reduction processing is aggregated to obtain the extended residual data.

[0172] For example, assuming that the image feature corresponding to the image residual data is y ∈ R {H,W,C}At this time, the image feature corresponding to the data in each group of image residual error groups can be expressed as y∈R {H ,W,C / 2} .

[0173] Step S103: Grouping the extended residual error data to obtain a plurality of extended residual error groups.

[0174] It should be noted that the grouping of the extended residual error data to obtain a plurality of extended residual error groups can be uniformly divided into a plurality of groups according to the corresponding feature channel. For example, assuming that the total number of feature channels corresponding to the extended residual error data is 20, at this time, the extended residual error data corresponding to the feature channels 1-10 can be divided into a group, and the extended residual error data corresponding to the feature channels 11-20 can be divided into a group. The number of uniformly divided groups can be set by the management personnel of the decoding device in advance, and the present embodiment does not limit this.

[0175] Of course, the grouping can also be non-uniform. The extended residual error data can be grouped based on the feature channel corresponding to the extended residual error data to obtain a plurality of extended residual error groups. The preset grouping rule can be set by the management personnel of the decoding device according to actual needs, for example, the preset grouping rule is set to divide the first m / n (n is the total number of feature channels, m is a preset value, and the value range is [1, n)) extended residual error data into a group, and the remaining extended residual error data into another group.

[0176] In actual use, when grouping the extended residual error data based on the feature channel corresponding to the extended residual error data to obtain a plurality of extended residual error groups, the extended residual error data corresponding to one feature channel can be divided into a group, for example, assuming that the total number of feature channels corresponding to the extended residual error data is 20, at this time, the extended residual error data can be divided into 20 groups according to the different feature channels.

[0177] Before grouping, the present embodiment will also detect whether the extracted is image residual error data or extended residual error data. If it is image residual error data, it will also be reduced in spatial resolution first to ensure that even if the image code stream transmitted by the encoding end contains image residual error data, it can still be normally grouped and processed after processing, improving the universality of the image decoding method of the present embodiment.

[0178] Reference Figure 6 , Figure 6 is a flowchart of a third embodiment of the image decoding method of the present application.

[0179] Based on the above first embodiment, the step S20 of the image decoding method of the present embodiment comprises:

[0180] Step S201: constructing a residual recovery sequence according to the plurality of extended residual groups.

[0181] It should be noted that after the plurality of extended residual groups are divided, residual recovery can be performed on the extended residual groups one by one, thereby reducing the time complexity of the mean prediction in the image decoding process. At this time, in order to determine the residual recovery order of each extended residual group, a residual recovery sequence can be constructed according to the plurality of extended residual groups.

[0182] Step S202: performing residual recovery on the plurality of extended residual groups based on the residual recovery sequence respectively, and obtaining image reconstruction features corresponding to each extended residual group.

[0183] It should be noted that performing residual recovery on the plurality of extended residual groups based on the residual recovery sequence can be sequentially performing residual recovery on the plurality of extended residual groups based on the sequence order in the residual recovery sequence.

[0184] In actual use, residual recovery can be sequentially performed in a sequence traversal manner, and at this time, the step S202 of the embodiment can include:

[0185] traversing the residual recovery sequence to obtain a current extended residual group;

[0186] obtaining auxiliary information output by the auxiliary encoding network;

[0187] constructing prior information according to the auxiliary information;

[0188] performing residual recovery on the current extended residual group based on the prior information, and obtaining image reconstruction features corresponding to the current extended residual group;

[0189] At the end of the traversal, image reconstruction features corresponding to each extended residual group are obtained.

[0190] It should be noted that traversing the residual recovery sequence to obtain a current extended residual group can be traversing the residual recovery sequence and taking the traversed extended residual group as the current extended residual group. The auxiliary encoding network can be the auxiliary network (Hyper Encoder Net or Hyper Decoder Net) shown in the above Figure 3 .

[0191] In actual use, the image reconstruction feature corresponding to the current extended residual group can be obtained by recovering the residual based on the prior information. The prior information can be processed by a convolution kernel to obtain a feature. The auxiliary information output by the auxiliary network and the obtained feature are spliced in a channel. After splicing, the spliced information is input into the Prediction Fusion Net to obtain a predicted mean. The predicted mean and the residual in the current extended residual group are added to realize residual recovery and obtain the image reconstruction feature corresponding to the current extended residual group.

[0192] In actual use, the step of constructing the prior information according to the auxiliary information can include:

[0193] The auxiliary information is processed to reduce the spatial resolution to obtain extended auxiliary information.

[0194] It is detected whether the current extended residual group is the first element in the residual recovery sequence.

[0195] If it is the first element, the prior information is constructed according to the extended auxiliary information.

[0196] If it is not the first element, the extended auxiliary information is spliced with the convolution processing result corresponding to the image reconstruction feature of the recovered extended residual group to obtain spliced auxiliary information, and the prior information is constructed according to the spliced auxiliary information.

[0197] It should be noted that the spatial size and the number of feature channels of the auxiliary information output by the auxiliary coding network are actually consistent with those of the original image. At this time, the extended residual data has been processed to reduce the spatial resolution. Therefore, at this time, in order to ensure smooth channel splicing, the auxiliary information needs to be processed to reduce the spatial resolution to obtain extended auxiliary information, and then the prior information is constructed according to the extended auxiliary information.

[0198] When the prior information is constructed according to the extended auxiliary information, in order to increase the accuracy of the mean prediction, the image feature corresponding to the reconstructed extended residual group can also be used to construct the prior information. If the current extended residual group is the first element in the residual recovery sequence, it means that the extended residual group is the first extended residual group to be recovered. At this time, there is no reconstructed extended residual group, so the prior information can be directly constructed according to the extended auxiliary information.

[0199] If the current extended residual group is not the first element in the residual recovery sequence, there is an already reconstructed extended residual group, and thus the image reconstruction feature of the recovered extended residual group can be convoluted by the convolution layer, the convolution result of the extended auxiliary information and the image reconstruction feature of the recovered extended residual group is spliced in the channel, and the prior information is constructed according to the spliced auxiliary information. When constructing the prior information, all the recovered extended residual groups can be selected, or only part of the recovered extended residual groups can be selected.

[0200] In a possible implementation of the embodiment, when the result is spliced, the image reconstruction feature of the recovered extended residual group can be enhanced in feature to further improve the prediction effect. The step of splicing the convolution result of the extended auxiliary information and the image reconstruction feature of the recovered extended residual group to obtain the spliced auxiliary information can include:

[0201] obtaining the image reconstruction feature corresponding to the recovered extended residual group;

[0202] enhancing the image reconstruction feature in feature to obtain an enhanced reconstruction feature;

[0203] splicing the convolution result of the auxiliary information and the enhanced reconstruction feature to obtain the spliced auxiliary information.

[0204] In actual use, the image reconstruction feature can be enhanced in feature to obtain an enhanced reconstruction feature, which can include missing value processing, abnormal value processing, etc.

[0205] It can be understood that, before splicing the auxiliary information and the image reconstruction feature, the image reconstruction feature is enhanced in feature to obtain an enhanced reconstruction feature, which can increase the reliability of the enhanced reconstruction feature, thereby improving the reliability of the prior information, and making the mean prediction more accurate according to the prior information.

[0206] In actual use, when the image reconstruction feature corresponding to the recovered extended residual group is enhanced in feature, the prediction mean value, auxiliary information, image residual data and / or residual data variance corresponding to the recovered extended residual group can be used, and thus the step of enhancing the image reconstruction feature in feature to obtain an enhanced reconstruction feature can include:

[0207] obtaining the prediction mean value, auxiliary information, image residual data and / or residual data variance corresponding to the recovered extended residual group;

[0208] The image reconstruction features are enhanced by using the predicted mean, auxiliary information, image residual data, and / or residual data variance corresponding to the recovered extended residual group to obtain enhanced reconstruction features.

[0209] It should be noted that the predicted mean corresponding to the recovered extended residual group can be the value obtained when predicting the mean during residual recovery of the recovered extended residual group. The variance of the residual data can be the variance value of the image residual data corresponding to the recovered extended residual group.

[0210] In one possible implementation of this embodiment, in order to improve the reconstruction effect of image reconstruction, step S30 of this embodiment may include:

[0211] Feature enhancement is performed on the image reconstruction features corresponding to each extended residual group to obtain the enhanced reconstruction features corresponding to each extended residual group;

[0212] The spatial resolution of the enhanced reconstruction features corresponding to each extended residual group is magnified to obtain the reconstruction feature data.

[0213] It should be noted that feature enhancement of the image reconstruction features corresponding to the extended residual group can be achieved by using the prediction mean, auxiliary information, image residual data and / or residual data variance of the extended residual group to enhance the image reconstruction features corresponding to the extended residual group.

[0214] It is understandable that before performing spatial resolution amplification processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data, feature enhancement is first performed on the image reconstruction features corresponding to each extended residual group, and then spatial resolution amplification processing is performed on the enhanced reconstruction features corresponding to each extended residual group to obtain reconstruction feature data. This can ensure that the reliability of the final constructed reconstruction feature data is higher, and the quality of the reconstructed image blocks obtained by subsequent image reconstruction will be better.

[0215] To facilitate understanding, we will now combine... Figure 7 , 8 The following points are provided for explanation, but do not limit the scope of this scheme. Figure 7 This is a schematic diagram of the image decoding grouping execution process in this embodiment. Figure 8 This is a schematic diagram of the secondary grouping execution process in this embodiment. Figure 9 This is a schematic diagram of the feature enhancement grouping execution process in this embodiment.

[0216] like Figure 7 As shown, the image features corresponding to the image residual data are y∈R {H,W,C} First, reduce the spatial resolution of the sample; the result is y∈R. {H / 2,W / 2,4C}At this time, it can be divided into two groups (group 1 and group 2), and then the residual recovery sequence (group 1-group 2) is constructed, and then the mean mu of the Group 1 point is obtained through the network by auxiliary (Psi) information, and the image reconstruction feature of the Group 1 point is obtained, and the image reconstruction feature of the Group 1 point is extracted, and the channel splicing is performed with Psi, and the mean mu of the Group 2 is obtained through the network, and the Group 2 is obtained by decoding. Combine the Group 1 point and the Group 2 to perform channel splicing, and then perform special upsampling spatial resolution processing (that is, the inverse process of downsampling spatial resolution processing), so as to obtain the reconstructed feature data, and then perform image reconstruction according to the reconstructed feature data.

[0217] If the grouping is performed before the downsampling spatial resolution processing, the execution process is as shown in Figure 8 The image feature corresponding to the image residual data is y∈R {H,W,C} At this time, y1 and y2 are divided according to the corresponding feature channels, and at this time, y1∈R {H,W,C / 2} At this time, the downsampling spatial resolution processing is performed on y1 and y2 respectively, y1 is divided into part1, part2, part3 and part4, and at this time, there are:

[0218] Part1 represents that in y1, the spatial position is in the even row and the even column of y1 spatial index, and contains data information of all channels.

[0219] Part2 represents that in y1, the spatial position is in the odd row and the odd column of y1 spatial index, and contains data information of all channels.

[0220] Part3 represents that in y1, the spatial position is in the even row and the odd column of y1 spatial index, and contains data information of all channels.

[0221] Part4 represents that in y1, the spatial position is in the odd row and the even column of y1 spatial index, and contains data information of all channels.

[0222] Similarly, y2 can also be divided into four parts, part5, part6, part7, part8, and the residual recovery sequence constructed at this time is "part1-part2-part3-part4-part5-part6-part7-part8", at this time, the prior information can be generated directly according to the auxiliary information when part1 is recovered, the prior information can be generated according to the image reconstruction features of part1 and auxiliary information when part2 is recovered, the prior information can be generated according to the image reconstruction features of part1 and part2 and auxiliary information when part3 is recovered, and so on until the image reconstruction features corresponding to all parts are obtained.

[0223] Of course, two residual recovery sequences can also be constructed according to y1 and y2 respectively, at this time, the two residual recovery sequences are "part1-part2-part3-part4" and "part5-part6-part7-part8", at this time, y1 can be reconstructed according to the residual recovery sequence composed of part1-4 by using the similar process as above to obtain the image reconstruction features corresponding to each part in y1, then the prior information is generated according to the image reconstruction features corresponding to y1 and auxiliary information, and y2 is reconstructed according to the residual recovery sequence composed of part5-8 and the generated prior information to obtain the image reconstruction features corresponding to each part in y2.

[0224] Among them, since y is first split into y1 and y2 and then processed by reducing the spatial resolution, after obtaining the image reconstruction features corresponding to each part in y1 and y2, the image reconstruction features corresponding to each part in y1 and y2 can be first processed by increasing the spatial resolution and then aggregated to obtain complete reconstruction feature data.

[0225] If the feature enhancement is performed after grouping, the specific execution process is as shown in Figure 9 a, the image feature corresponding to the image residual data is y∈R {H,W,C} , which is first processed by reducing the spatial resolution to obtain y∈R {H / 2,W / 2,4C}At this time, the image feature data can be divided into two groups (group 1 and group 2), and then the mean value mu of the group 1 points is obtained through the network by means of the auxiliary (Psi) information, the image reconstruction feature of the group 1 is obtained, the image reconstruction feature of the group 1 is enhanced by using the corresponding prediction mean value of the group 1, the enhanced image feature group 1_E is obtained, and then the feature of the enhanced image feature is extracted and spliced with the Psi channel, the mean value mu of the group 2 is obtained through the network, and then the image reconstruction feature of the group 2 is decoded to obtain the image reconstruction feature of the group 2, and then the image reconstruction feature of the group 2 is enhanced according to the mean value mu of the group 2 to obtain the enhanced image feature group 2_E. The channel splicing of the group 1_E and the group 2_E is performed, and then a special magnification spatial resolution processing (that is, the inverse process of the reduction spatial resolution processing) is performed, so that the reconstruction feature data is obtained, and then the image reconstruction is performed according to the reconstruction feature data. In the process of enhancing the feature in the group 1, the network (Enhance_Net) used for the feature enhancement has a specific structure as shown in Figure 9 a, Figure 9

[0226] In the embodiment, the residual recovery sequence is constructed according to the plurality of extended residual groups, and the residual recovery is performed on the plurality of extended residual groups based on the residual recovery sequence, so that the image reconstruction features corresponding to the plurality of extended residual groups are obtained. Since the residual recovery sequence is constructed according to the plurality of extended residual groups, the order of the residual recovery can be determined through the residual recovery sequence, so that it can be quickly determined whether there is an already recovered extended residual group, and when there is an already recovered extended residual group, more accurate prior information can be constructed according to the image feature data corresponding to the already recovered extended residual group.

[0227] The embodiment of the present application provides an image coding method, which refers to Figure 10 , Figure 10 FIG. 1 is a flowchart of an image coding method according to the first embodiment of the present application.

[0228] In the embodiment, the image coding method comprises the following steps:

[0229] Step S100: performing reduction spatial resolution processing on the image feature corresponding to the to-be-coded image to obtain extended image feature data.

[0230] It should be noted that, in order to facilitate subsequent grouping processing and reduce the time complexity, after the image feature corresponding to the to-be-coded image is obtained, the image residual data can be subjected to reduction spatial resolution processing to obtain the extended image feature. The to-be-coded image is the original image mentioned in the image decoding method embodiment.

[0231] ​Wherein, after the image feature is processed by reducing the spatial resolution, the corresponding spatial size will be smaller, and a smaller convolution kernel can be used when processing the image feature. For example, if a 5x5 convolution kernel is used on the original image residual data, it is equivalent to using a 3x3 convolution kernel on the image feature after the image feature is processed by reducing the spatial resolution.

[0232] In a possible implementation of the present embodiment, the step S100 can include:

[0233] reducing the spatial size of the image feature of the image to be encoded, and / or increasing the number of feature channels corresponding to the image feature, to obtain an expanded image feature.

[0234] It should be noted that the reduction amplitude of the spatial size of the image feature and the increase amplitude of the number of feature channels corresponding to the image feature can be set by the management personnel of the decoding device in advance, and the present embodiment does not limit this.

[0235] Wherein, in actual execution, only the reduction amplitude of the spatial size can be set, or only the increase amplitude of the number of feature channels can be set, the decoding device can adjust the spatial size or the number of feature channels adaptively, of course, the decoding device can also be set with the reduction amplitude of the spatial size and the increase amplitude of the number of feature channels, and then the decoding device can be executed.

[0236] In a specific implementation, the image feature can be processed by reducing the spatial resolution based on the spatial information or the frequency information corresponding to the image feature, and the image feature can also be processed by reducing the spatial resolution through a preset convolution layer. At this time, the step of reducing the spatial size of the image feature of the image to be encoded and / or increasing the number of feature channels corresponding to the image feature, to obtain an expanded image feature, can include:

[0237] reducing the spatial size of the image feature of the image to be encoded based on the spatial information corresponding to the image feature, and / or increasing the number of feature channels corresponding to the image feature, to obtain expanded residual data;

[0238] or,

[0239] reducing the spatial size of the image feature of the image to be encoded based on the frequency information corresponding to the image feature, and / or increasing the number of feature channels corresponding to the image feature, to obtain expanded residual data

[0240] or,

[0241] The spatial size corresponding to the image feature is reduced and / or the number of feature channels corresponding to the image feature is increased according to a preset convolutional layer, to obtain extended residual data.

[0242] In the specific implementation, reference can be made to the description of the image decoding method embodiments above. Figure 5

[0243] Step S200: Grouping the extended image features to obtain a plurality of extended feature groups.

[0244] In the specific implementation, when the extended image features are grouped, reference can be made to the feature channels corresponding to the extended image features, and thus the step S200 in this embodiment can include:

[0245] The extended image features are grouped based on the feature channels corresponding to the extended feature data to obtain a plurality of extended residual groups.

[0246] It should be noted that grouping the extended image features based on the feature channels corresponding to the extended feature data to obtain a plurality of extended residual groups can be to divide the extended image features into a plurality of groups uniformly according to the corresponding feature channels. For example, assuming that the total number of feature channels corresponding to the extended image features is 20, the extended image features corresponding to the feature channels 1-10 can be divided into a group, and the extended image features corresponding to the feature channels 11-20 can be divided into a group. The number of groups in the uniform division can be set by the administrator of the decoding device in advance, and this embodiment does not limit this.

[0247] Of course, the grouping can also be non-uniform, and grouping the extended image features based on the feature channels corresponding to the extended image features to obtain a plurality of extended residual groups can also be to divide the extended image features into a plurality of groups according to the corresponding feature channels based on a preset grouping rule. The preset grouping rule can be set by the administrator of the encoding device according to actual needs in advance, for example, the preset grouping rule is set to divide the first m / n (n is the total number of feature channels, and m is a preset value, the value range is [1, n)) extended image features into a group, and the remaining extended image features into another group.

[0248] In actual use, when the extended residual data is grouped based on the feature channels corresponding to the extended image features to obtain a plurality of extended residual groups, the extended image features corresponding to one feature channel can also be divided into a group, for example, assuming that the total number of feature channels corresponding to the extended image features is 20, the extended image features can be divided into 20 groups according to the different feature channels.

[0249] Step S300: Residual calculation is performed on the plurality of extended feature groups respectively to obtain image residual data corresponding to each extended feature group.​

[0250] It should be noted that the residual calculation of the extended feature group can be mean prediction of the extended feature group, and then the image features in the extended feature group are subtracted from the predicted mean to obtain the image residual data corresponding to the extended feature group.

[0251] Step S400: generating an image code stream according to the image residual data, and sending the image code stream to an image decoding end.

[0252] It should be noted that the generation of the image code stream according to the image residual data can be writing the image residual data into the image code stream through entropy coding.

[0253] In a possible implementation manner of the embodiment, in the reduction of the spatial resolution of the image features corresponding to the image to be encoded, the image features can be grouped first, and then the reduction of the spatial resolution is performed respectively. At this time, the step S100 of the embodiment can include:

[0254] Obtaining image features corresponding to an image to be encoded;

[0255] Grouping the image features according to feature channels corresponding to the image features, to obtain at least one image feature group;

[0256] Performing reduction of the spatial resolution on the data in the image feature group to obtain extended image features.

[0257] It should be noted that the image features corresponding to the image to be encoded can be extracted by a preset feature extraction network. The same or similar manner as that for grouping the extended image features can be used when the data is grouped according to the feature channels corresponding to the feature channels of the image features.

[0258] In a possible implementation manner of the embodiment, the step S400 of the embodiment can include:

[0259] Performing magnification of the spatial resolution on the image reconstruction features corresponding to each extended feature group to obtain image residual data corresponding to the image to be encoded;

[0260] Generating an image code stream according to the image residual data corresponding to the image to be encoded, and sending the image code stream to an image decoding end.

[0261] It should be noted that the enlarged spatial resolution processing can be the inverse process of the above-mentioned reduced spatial resolution processing. After obtaining the image reconstruction features corresponding to each extended feature group, the image reconstruction features corresponding to each extended feature group can be subjected to enlarged spatial resolution processing to restore the spatial size and the number of channels thereof to be consistent with the image features corresponding to the image to be encoded, so as to obtain the image residual data corresponding to the image to be encoded. Then, the image residual data corresponding to the image to be encoded is subjected to entropy coding, the image code stream is generated, and the generated image code stream is sent to the image decoding end.

[0262] The embodiment obtains extended image features by performing reduced spatial resolution processing on the image features corresponding to the image to be encoded, groups the extended image features to obtain a plurality of extended feature groups, respectively performs residual calculation on the plurality of extended feature groups to obtain image residual data corresponding to each extended feature group, generates an image code stream according to the image residual data, and sends the image code stream to the image decoding end. Since the image features of the image to be encoded are subjected to reduced spatial resolution processing and then grouped into a plurality of extended residual groups, the residual calculation processing can be performed on the entire group at a low resolution, thereby improving the overall residual calculation efficiency and reducing the time complexity.

[0263] Reference Figure 11 , Figure 11 The flowchart of the second embodiment of the image encoding method of the present application is shown.

[0264] Based on the above-mentioned first embodiment, the step S300 of the image encoding method of the present embodiment comprises:

[0265] Step S3001: constructing a residual calculation sequence according to the plurality of extended feature groups.

[0266] It should be noted that after the plurality of extended feature groups are divided, residual calculation can be performed on the entire group. At this time, in order to determine the residual calculation sequence of each extended feature group, a residual calculation sequence can be constructed according to the plurality of extended feature groups.

[0267] Step S3002: respectively performing residual calculation on the plurality of extended feature groups based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group.

[0268] It should be noted that respectively performing residual calculation on the plurality of extended feature groups based on the residual calculation sequence can be sequentially performing residual calculation on the plurality of extended feature groups based on the sequence order in the residual calculation sequence.

[0269] In actual use, residual calculation can be sequentially performed through sequence traversal. At this time, the step S3002 of the present embodiment can comprise:

[0270] traversing the residual calculation sequence to obtain a current extended feature group;

[0271] obtaining auxiliary information output by the auxiliary coding network;

[0272] constructing prior information according to the auxiliary information;

[0273] performing residual calculation on the current extended feature group based on the prior information to obtain image residual data corresponding to the current extended feature group;

[0274] obtaining image residual data corresponding to each extended feature group at the end of the traversal.

[0275] It should be noted that the current extended feature group can be obtained by traversing the residual calculation sequence and taking the traversed extended feature group as the current extended feature group. The auxiliary coding network can be the auxiliary network (Hyper Encoder Net or Hyper Decoder Net) described in the above Figure 3 .

[0276] In actual use, the residual calculation on the current extended feature group based on the prior information to obtain image residual data corresponding to the current extended feature group can be that the prior information is processed by a convolution kernel to obtain a feature, the auxiliary information output by the auxiliary network is channel-spliced with the obtained feature, and then the spliced result is input into the Prediction Fusion Net to obtain a predicted mean value, and then the feature in the current extended feature group is subtracted from the predicted mean value to realize residual calculation and obtain image residual data corresponding to the current extended feature group.

[0277] When constructing the prior information according to the auxiliary information, in order to increase the accuracy of the mean value prediction, the image features corresponding to the extended feature groups that have been subjected to residual calculation can also be used to construct the prior information, and the specific implementation manner is the same as that applied in the image decoding process. The specific implementation steps can refer to the manner of constructing the prior information according to the auxiliary information provided in any of the above image decoding method embodiments.

[0278] The present embodiment constructs a residual calculation sequence according to a plurality of extended feature groups, and performs residual calculation on the plurality of extended feature groups based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group. Since the residual calculation sequence is constructed according to the plurality of extended feature groups, the order of residual calculation can be determined through the residual calculation sequence, so that it can be quickly determined whether there is an already-calculated extended feature group, and when there is an already-calculated extended feature group, more accurate prior information can be constructed according to the image feature data corresponding to the already-calculated extended feature group.

[0279] Further, the present application also provides a storage medium, wherein the storage medium stores an image decoding program and / or an image encoding program, the image decoding program is executed by a processor to implement the steps of the image decoding method as described above, and the image encoding program is executed by a processor to implement the steps of the image encoding method as described above.

[0280] With reference to Figure 12 , Figure 12 FIG. 1 is a structural block diagram of an image decoding device according to an embodiment of the present application.

[0281] As shown in Figure 12 , the image decoding device according to an embodiment of the present application comprises:

[0282] a code stream decoding module 10 configured to group the extracted extended residual data to obtain a plurality of extended residual groups;

[0283] a residual recovery module 20 configured to perform residual recovery on the plurality of extended residual groups respectively to obtain image reconstruction features corresponding to the extended residual groups;

[0284] a data combination module 30 configured to perform upscaling spatial resolution processing on the image reconstruction features corresponding to the extended residual groups to obtain reconstruction feature data;

[0285] an image reconstruction module 40 configured to perform image reconstruction according to the reconstruction feature data to obtain a reconstructed image block.

[0286] According to the present application, the extracted extended residual data is grouped to obtain a plurality of extended residual groups, the plurality of extended residual groups are subjected to residual recovery respectively to obtain image reconstruction features corresponding to the extended residual groups, the image reconstruction features corresponding to the extended residual groups are subjected to upscaling spatial resolution processing to obtain reconstruction feature data, and image reconstruction is performed according to the reconstruction feature data to obtain a reconstructed image block. Since the obtained extended residual data is residual data subjected to downscaling spatial resolution processing, the residual recovery processing can be performed on the grouped extended residual data at a low resolution, thereby improving the overall residual recovery calculation efficiency and reducing the time complexity.

[0287] In a possible implementation manner of the present application, the code stream decoding module 10 is further configured to extract the extended residual data from the image code stream, and group the extended residual data based on the feature channels corresponding to the extended residual data to obtain a plurality of extended residual groups.

[0288] In a possible implementation of the present embodiment, the code stream decoding module 10 is further configured to: extract image residual data from the image code stream; perform down-spatial-resolution processing on the image residual data to obtain extended residual data, the up-spatial-resolution processing being inverse processing of the down-spatial-resolution processing; and group the extended residual data to obtain a plurality of extended residual groups.

[0289] In a possible implementation of the present embodiment, the code stream decoding module 10 is further configured to: group data in the image residual data according to a feature channel corresponding to the image residual data to obtain at least one image residual group; and perform down-spatial-resolution processing on the data in the image residual group to obtain extended residual data.

[0290] In a possible implementation of the present embodiment, the code stream decoding module 10 is further configured to: reduce a spatial size corresponding to the image residual data, and / or increase a number of feature channels corresponding to the image residual data, to obtain the extended residual data.

[0291] In a possible implementation of the present embodiment, the code stream decoding module 10 is further configured to: reduce the spatial size corresponding to the image residual data according to spatial information corresponding to the image residual data, and / or increase the number of feature channels corresponding to the image residual data, to obtain the extended residual data; or reduce the spatial size corresponding to the image residual data according to frequency information corresponding to the image residual data, and / or increase the number of feature channels corresponding to the image residual data, to obtain the extended residual data; or reduce the spatial size corresponding to the image residual data according to a preset convolution layer, and / or increase the number of feature channels corresponding to the image residual data, to obtain the extended residual data.

[0292] In a possible implementation of the present embodiment, the residual recovery module 20 is further configured to: construct a residual recovery sequence according to the plurality of extended residual groups; and perform residual recovery on the plurality of extended residual groups based on the residual recovery sequence to obtain image reconstruction features corresponding to each of the extended residual groups.

[0293] In a possible implementation of the present embodiment, the residual recovery module 20 is further configured to: traverse the residual recovery sequence to obtain a current extended residual group; obtain auxiliary information output by an auxiliary coding network; construct prior information according to the auxiliary information; perform residual recovery on the current extended residual group based on the prior information to obtain image reconstruction features corresponding to the current extended residual group; and obtain image reconstruction features corresponding to each of the extended residual groups at the end of the traversal.

[0294] In a possible implementation of the embodiment, the residual recovery module 20 is further configured to perform down-spatial resolution processing on the auxiliary information to obtain extended auxiliary information; detect whether the current extended residual group is the first element in the residual recovery sequence; if the current extended residual group is the first element, construct prior information according to the extended auxiliary information; if the current extended residual group is not the first element, splice the extended auxiliary information and a convolution processing result corresponding to the image reconstruction feature of the recovered extended residual group to obtain spliced auxiliary information, and construct prior information according to the spliced auxiliary information.

[0295] In a possible implementation of the embodiment, the residual recovery module 20 is further configured to obtain the image reconstruction feature corresponding to the recovered extended residual group; perform feature enhancement on the image reconstruction feature to obtain enhanced reconstruction feature; and splice the auxiliary information and a convolution processing result corresponding to the enhanced reconstruction feature to obtain spliced auxiliary information.

[0296] In a possible implementation of the embodiment, the residual recovery module 20 is further configured to obtain the predicted mean value, the auxiliary information, the image residual data and / or the residual data variance corresponding to the recovered extended residual group; perform feature enhancement on the image reconstruction feature according to the predicted mean value, the auxiliary information, the image residual data and / or the residual data variance corresponding to the recovered extended residual group to obtain enhanced reconstruction feature.

[0297] In a possible implementation of the embodiment, the data combination module 30 is further configured to perform feature enhancement on the image reconstruction feature corresponding to each extended residual group to obtain enhanced reconstruction feature corresponding to each extended residual group; and perform up-spatial resolution processing on the enhanced reconstruction feature corresponding to each extended residual group to obtain reconstruction feature data.

[0298] Referring to Figure 13 , Figure 13 FIG. 1 is a structural block diagram of an image coding device according to an embodiment of the present application.

[0299] As shown in Figure 13 , the image coding device according to the embodiment of the present application comprises:

[0300] a feature extraction module 100 configured to perform down-spatial resolution processing on image features corresponding to a to-be-coded image to obtain extended image features;

[0301] a data grouping module 200 configured to group the extended image features to obtain a plurality of extended feature groups;

[0302] a residual calculation module 300 configured to perform residual calculation on the plurality of extended feature groups respectively to obtain image residual data corresponding to each extended feature group;

[0303] The code stream generation module 400 is configured to generate an image code stream according to the image residual data, and send the image code stream to an image decoding end.

[0304] In the embodiment, the image feature corresponding to the image to be encoded is subjected to the reduced spatial resolution processing to obtain an extended image feature; the extended image feature is grouped to obtain a plurality of extended feature groups; the plurality of extended feature groups are subjected to residual calculation respectively to obtain image residual data corresponding to each extended feature group; and the image code stream is generated according to the image residual data and sent to the image decoding end. Since the image feature of the image to be encoded is subjected to the reduced spatial resolution processing and then grouped into a plurality of extended residual groups, the residual calculation of the whole group at a low resolution can be implemented, thereby improving the overall residual calculation efficiency and reducing the time complexity.

[0305] In a possible implementation manner of the embodiment, the feature extraction module 100 is further configured to obtain an image feature corresponding to an image to be encoded; group the image feature according to a feature channel corresponding to the image feature to obtain at least one image feature group; and perform reduced spatial resolution processing on data in the image feature group to obtain an extended image feature.

[0306] In a possible implementation manner of the embodiment, the feature extraction module 100 is further configured to

[0307] The spatial size corresponding to the image feature of the image to be encoded is reduced, and / or the number of feature channels corresponding to the image feature is increased to obtain the extended image feature.

[0308] In a possible implementation manner of the embodiment, the feature extraction module 100 is further configured to reduce the spatial size corresponding to the image feature according to spatial information corresponding to the image feature of the image to be encoded, and / or increase the number of feature channels corresponding to the image feature to obtain the extended residual data; or reduce the spatial size corresponding to the image feature according to frequency information corresponding to the image feature of the image to be encoded, and / or increase the number of feature channels corresponding to the image feature to obtain the extended residual data; or reduce the spatial size corresponding to the image feature according to a preset convolution layer, and / or increase the number of feature channels corresponding to the image feature to obtain the extended residual data.

[0309] In a possible implementation manner of the embodiment, the residual calculation module 300 is further configured to construct a residual calculation sequence according to the plurality of extended feature groups; and perform residual calculation on the plurality of extended feature groups respectively based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group.

[0310] In a possible implementation of the embodiment, the residual calculation module 300 is further configured to traverse the residual calculation sequence to obtain a current extended feature group; obtain auxiliary information output by an auxiliary coding network; construct prior information according to the auxiliary information; perform residual calculation on the current extended feature group based on the prior information to obtain image residual data corresponding to the current extended feature group; and obtain image residual data corresponding to each extended feature group at the end of the traversal.

[0311] In a possible implementation of the embodiment, the data grouping module 200 is further configured to group the extended image features based on feature channels corresponding to the extended feature data to obtain a plurality of extended residual groups.

[0312] In a possible implementation of the embodiment, the code stream generation module 400 is further configured to perform amplification spatial resolution processing on image reconstruction features corresponding to each extended feature group to obtain image residual data corresponding to a to-be-coded image, the amplification spatial resolution processing being an inverse process of the reduction spatial resolution processing; generate an image code stream according to the image residual data corresponding to the to-be-coded image, and send the image code stream to an image decoding end.

[0313] It should be understood that the above is only for illustration, and does not constitute any limitation on the technical solutions of the present application. In specific applications, those skilled in the art can set it up according to the needs, and the present application does not limit it.

[0314] It should be noted that the above-described workflow is only illustrative and does not limit the scope of protection of the present application. In actual applications, those skilled in the art can select part or all of them to achieve the purpose of the embodiment, and this place does not limit it.

[0315] In addition, technical details not described in detail in the embodiment can be referred to the image decoding method or image encoding method provided by any embodiment of the present application, which will not be described here.

[0316] In addition, it should be noted that in this document, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of another identical element in the process, method, article or system that includes the element.

[0317] The above embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0318] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, also can be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of contribution to the prior art can be embodied in the form of software product, the computer software product is stored in a storage medium (such as read only memory (Read Only Memory, ROM) / RAM, disk, optical disk), including a number of instructions to make a terminal device (may be a mobile phone, computer, server, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0319] The above is only the preferred embodiment of the present application, not therefore limit the patent scope of the present application, any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image decoding method, characterized in that, The image decoding method includes: Image residual data or extended residual data is extracted from the image bitstream, and multiple extended residual groups are obtained based on the extracted image residual data or extended residual data; wherein, the extended residual data is obtained by reducing the spatial resolution of the image residual data; Residual recovery is performed on each of the multiple extended residual groups to obtain image reconstruction features corresponding to each extended residual group; wherein, a residual recovery sequence is constructed based on the multiple extended residual groups; residual recovery is performed on each of the multiple extended residual groups based on the residual recovery sequence to obtain image reconstruction features corresponding to each extended residual group; The image reconstruction features corresponding to each extended residual group are processed to increase the spatial resolution to obtain reconstruction feature data. Image reconstruction is performed based on the reconstructed feature data to obtain reconstructed image blocks.

2. The method as described in claim 1, characterized in that, The step of extracting image residual data or extended residual data from the image bitstream, and obtaining multiple extended residual groups based on the extracted image residual data or extended residual data, includes: Extract the image residual data from the image bitstream; The image residual data is processed to reduce the spatial resolution to obtain the extended residual data. The process of increasing the spatial resolution is the inverse process of reducing the spatial resolution. The extended residual data is grouped to obtain multiple extended residual groups.

3. The method as described in claim 2, characterized in that, The step of reducing the spatial resolution of the image residual data to obtain the extended residual data includes: The spatial size corresponding to the image residual data is reduced, and / or the number of feature channels corresponding to the image residual data is increased to obtain the extended residual data.

4. The method as described in claim 3, characterized in that, The step of reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain the extended residual data includes: The spatial size of the image residual data is reduced and / or the number of feature channels corresponding to the image residual data is increased based on the spatial information corresponding to the image residual data to obtain the extended residual data.

5. The method as described in claim 1, characterized in that, The step of performing residual recovery on the plurality of extended residual groups based on the residual recovery sequence to obtain the image reconstruction features corresponding to each extended residual group includes: The residual recovery sequence is traversed to obtain the current extended residual set; Obtain prior information; Based on the prior information, residual recovery is performed on the current extended residual group to obtain the image reconstruction features corresponding to the current extended residual group; At the end of the traversal, the image reconstruction features corresponding to each extended residual group are obtained.

6. The method as described in claim 5, characterized in that, The acquisition of prior information includes: Extended auxiliary information is obtained based on the information output by the auxiliary coding network; Detect whether the current extended residual group is the first element in the residual recovery sequence; If the current extended residual group is the first element, then prior information is constructed based on the extended auxiliary information; If the current extended residual group is not the first element, the extended auxiliary information is concatenated with the convolution processing result corresponding to the image reconstruction feature of the recovered extended residual group to obtain concatenation auxiliary information, and prior information is constructed based on the concatenation auxiliary information.

7. The method according to claim 1, characterized in that, The step of reconstructing the image based on the reconstructed feature data to obtain reconstructed image patches includes: The image reconstruction is achieved by performing synthetic transformation processing on the reconstructed feature data through a pre-constructed synthetic transformation network to obtain the reconstructed image patch. The synthetic transformation network is a network built based on deep learning or neural networks.

8. The image decoding method according to claim 2, characterized in that, The step of grouping the extended residual data to obtain multiple extended residual groups includes: The extended residual data is grouped evenly according to the feature channels corresponding to the extended residual data.

9. An image encoding method, characterized in that, The image encoding method includes: The spatial resolution of the image features corresponding to the image to be encoded is reduced to obtain extended image features. The extended image features are grouped to obtain multiple extended feature groups; Residual calculations are performed on the multiple extended feature groups respectively to obtain image residual data corresponding to each extended feature group; wherein, a residual calculation sequence is constructed based on the multiple extended feature groups; residual calculations are performed on the multiple extended feature groups based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group; An image bitstream is generated based on the image residual data, and the image bitstream is sent to the image decoding end.

10. An image decoding device, characterized in that, The image decoding device includes: The bitstream decoding module is used to extract image residual data or extended residual data from the image bitstream, and obtain multiple extended residual groups based on the extracted image residual data or extended residual data; wherein, the extended residual data is obtained by reducing the spatial resolution of the image residual data; The residual recovery module is used to perform residual recovery on the plurality of extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group; wherein, the residual recovery module is specifically used to: construct a residual recovery sequence based on the plurality of extended residual groups; and perform residual recovery on the plurality of extended residual groups respectively based on the residual recovery sequence to obtain image reconstruction features corresponding to each extended residual group; The data combination module is used to enlarge the spatial resolution of the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data. The image reconstruction module is used to reconstruct the image based on the reconstruction feature data to obtain reconstructed image blocks.

11. An image encoding device, characterized in that, The image encoding device includes: The feature extraction module is used to reduce the spatial resolution of the image features corresponding to the image to be encoded, thereby obtaining extended image features. The data grouping module is used to group the extended image features to obtain multiple extended feature groups; The residual calculation module is used to perform residual calculations on the plurality of extended feature groups respectively to obtain image residual data corresponding to each extended feature group; wherein, the residual calculation module is specifically used to: construct a residual calculation sequence based on the plurality of extended feature groups; and perform residual calculations on the plurality of extended feature groups respectively based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group; The bitstream generation module is used to generate an image bitstream based on the image residual data and send the image bitstream to the image decoding end.

12. A decoding device, characterized in that, The decoding device includes: a processor, a memory, and an image decoding program stored in the memory and executable on the processor, wherein the image decoding program, when executed by the processor, implements the steps of the image decoding method as described in any one of claims 1-8.

13. An encoding device, characterized in that, The encoding device includes: a processor, a memory, and an image decoding program and / or an image encoding program stored in the memory and executable on the processor. When the image decoding program is executed by the processor, it implements the steps of the image decoding method as described in any one of claims 1-8, and when the image encoding program is executed by the processor, it implements the steps of the image encoding method as described in claim 9.

14. A storage medium, characterized in that, The storage medium stores an image decoding program and / or an image encoding program. When the image decoding program is executed, it implements the steps of the image decoding method as described in any one of claims 1-8. When the image encoding program is executed, it implements the steps of the image encoding method as described in claim 9.

Images