Image decoding and coding method and device, equipment and storage medium
By performing spatial resolution processing and grouping of image code streams, the average prediction time complexity in the image encoding and decoding process is reduced, and the efficiency of image decoding and coding is improved.
Patent Information
- Application Number
- CN202411379207.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-31
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-03-31
AI Technical Summary
In the prior art, the average prediction time complexity in the image encoding and decoding process is high, resulting in a high overall execution time complexity.
By extracting image residual data or extending residual data from the image code stream, reducing and amplifying the spatial resolution processing, residual recovery and residual calculation are performed after grouping, reducing time complexity.
The overall residual recovery calculation efficiency is improved, the time complexity is reduced, and efficient image decoding and encoding at low resolution is achieved.
Smart Images

Figure CN118972620A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an image decoding and encoding method, device, equipment and storage medium. Background Art
[0002] In image compression schemes based on deep learning, the current mainstream approach is to use the decoded feature points as prior information to perform mean prediction on the feature points currently being decoded in order to reduce the spatial redundancy of the image. Mainstream schemes generally use serial or wave-front methods for encoding and decoding, and the serialization increases with the increase of feature resolution, resulting in a high overall execution time complexity.
[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention
[0004] The main purpose of the present invention is to provide an image decoding and encoding method, device, equipment and storage medium, aiming to solve the technical problem of high time complexity of mean prediction in the image encoding and decoding process of the prior art.
[0005] To achieve the above object, the present invention provides an image decoding method, which comprises the following steps:
[0006] Extracting image residual data or extended residual data from an image code stream, and obtaining a plurality of extended residual groups based on the extracted image residual data or the extended residual data;
[0007] Performing residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group;
[0008] Amplifying the spatial resolution of the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data;
[0009] Image reconstruction is performed according to the reconstruction feature data to obtain a reconstructed image block.
[0010] In a possible implementation manner of the present application, extracting image residual data or extended residual data from an image code stream, and obtaining a plurality of extended residual groups based on the extracted image residual data or the extended residual data, includes:
[0011] Extracting the image residual data from the image code stream;
[0012] Performing spatial domain resolution reduction processing on the image residual data to obtain the extended residual data, wherein the spatial domain resolution enlargement processing is the inverse process of the spatial domain resolution reduction processing;
[0013] The extended residual data are grouped to obtain a plurality of extended residual groups.
[0014] In a possible implementation manner of the present application, the reducing the spatial resolution of the image residual data to obtain the extended residual data includes:
[0015] The spatial size corresponding to the image residual data is reduced, and / or the number of feature channels corresponding to the image residual data is increased to obtain extended residual data.
[0016] In a possible implementation manner of the present application, reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain the extended residual data includes:
[0017] The spatial size corresponding to the image residual data is reduced according to the spatial domain information corresponding to the image residual data, and / or the number of feature channels corresponding to the image residual data is increased to obtain the extended residual data.
[0018] In a possible implementation manner of the present application, performing residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group includes:
[0019] constructing a residual recovery sequence according to the multiple extended residual groups;
[0020] Based on the residual recovery sequence, residual recovery is performed on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group.
[0021] In a possible implementation manner of the present application, the step of performing residual recovery on the multiple extended residual groups based on the residual recovery sequence to obtain image reconstruction features corresponding to each extended residual group includes:
[0022] Traversing the residual recovery sequence to obtain a current extended residual group;
[0023] Obtain auxiliary information output by the auxiliary encoding network;
[0024] constructing prior information based on the auxiliary information;
[0025] Performing residual recovery on the current extended residual group based on the prior information to obtain image reconstruction features corresponding to the current extended residual group;
[0026] At the end of the traversal, the image reconstruction features corresponding to each extended residual group are obtained.
[0027] In a possible implementation manner of the present application, constructing the prior information according to the auxiliary information includes:
[0028] Get extended auxiliary information;
[0029] Detecting whether the current extended residual group is the first element in the residual recovery sequence;
[0030] If it is the first element, constructing the prior information according to the extended auxiliary information;
[0031] If it is not the first element, the extended auxiliary information is spliced with the convolution processing result corresponding to the image reconstruction feature of the restored extended residual group to obtain spliced auxiliary information, and prior information is constructed according to the spliced auxiliary information.
[0032] In addition, to achieve the above object, the present invention also proposes an image encoding method, the image encoding method comprising:
[0033] Performing spatial domain resolution reduction processing on image features corresponding to the image to be encoded to obtain extended image features;
[0034] Grouping the extended image features to obtain a plurality of extended feature groups;
[0035] Performing residual calculations on the multiple extended feature groups respectively to obtain image residual data corresponding to each extended feature group;
[0036] An image code stream is generated according to the image residual data, and the image code stream is sent to an image decoding end.
[0037] In a possible implementation manner of the present application, the reducing the spatial resolution of the image features corresponding to the image to be encoded to obtain the extended image features includes:
[0038] Obtain image features corresponding to the image to be encoded;
[0039] The spatial resolution of the data in the image feature is reduced to obtain an extended image feature.
[0040] In a possible implementation manner of the present application, the reducing the spatial resolution of the image features corresponding to the image to be encoded to obtain the extended image features includes:
[0041] The spatial size corresponding to the image feature of the image to be encoded is reduced, and / or the number of feature channels corresponding to the image feature is increased to obtain the extended image feature.
[0042] In a possible implementation manner of the present application, reducing the spatial size corresponding to the image feature of the image to be encoded, and / or increasing the number of feature channels corresponding to the image feature to obtain the extended image feature, includes:
[0043] According to the spatial domain information corresponding to the image feature of the image to be encoded, the spatial size corresponding to the image feature is reduced, and / or the number of feature channels corresponding to the image feature is increased to obtain an extended feature group.
[0044] In a possible implementation manner of the present application, performing residual calculations on the multiple extended feature groups respectively to obtain image residual data corresponding to each extended feature group includes:
[0045] constructing a residual calculation sequence according to the plurality of extended feature groups;
[0046] Residual calculations are performed on the multiple extended feature groups respectively based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group.
[0047] In a possible implementation manner of the present application, performing residual calculations on the multiple extended feature groups based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group includes:
[0048] Traversing the residual calculation sequence to obtain a current extended feature group;
[0049] Obtain auxiliary information output by the auxiliary encoding network;
[0050] constructing prior information based on the auxiliary information;
[0051] Performing residual calculation on the current extended feature group based on the prior information to obtain image residual data corresponding to the current extended feature group;
[0052] At the end of the traversal, the image residual data corresponding to each extended feature group is obtained.
[0053] In a possible implementation manner of the present application, grouping the extended image features to obtain a plurality of extended feature groups includes:
[0054] The extended image features are grouped based on feature channels corresponding to the extended feature data to obtain a plurality of extended feature groups.
[0055] In a possible implementation manner of the present application, generating an image code stream according to the image residual data, and sending the image code stream to an image decoding end, includes:
[0056] Performing a spatial resolution enlargement process on the image reconstruction features corresponding to each extended feature group to obtain image residual data corresponding to the image to be encoded, wherein the spatial resolution enlargement process is the inverse process of the spatial resolution reduction process;
[0057] An image code stream is generated according to the image residual data corresponding to the image to be encoded, and the image code stream is sent to an image decoding end.
[0058] In addition, to achieve the above object, the present invention further provides an image decoding device, the image decoding device comprising:
[0059] A code stream decoding module, used to extract image residual data or extended residual data from the image code stream, and obtain a plurality of extended residual groups based on the extracted image residual data or the extended residual data;
[0060] A residual recovery module, used to perform residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group;
[0061] A data combination module is used to enlarge the spatial resolution of the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data;
[0062] The image reconstruction module is used to perform image reconstruction according to the reconstruction feature data to obtain a reconstructed image block.
[0063] In addition, to achieve the above object, the present invention further provides an image encoding device, the image encoding device comprising:
[0064] A feature extraction module is used to reduce the spatial resolution of the image features corresponding to the image to be encoded to obtain extended image features;
[0065] A data grouping module, used for grouping the extended image features to obtain a plurality of extended feature groups;
[0066] A residual calculation module, used to perform residual calculations on the multiple extended feature groups respectively to obtain image residual data corresponding to each extended feature group;
[0067] The code stream generation module is used to generate an image code stream according to the image residual data, and send the image code stream to the image decoding end.
[0068] In addition, to achieve the above-mentioned purpose, the present invention also proposes a decoding device, which includes: a processor, a memory, and an image decoding program stored in the memory and executable on the processor, and the image decoding program implements the steps of the image decoding method described above when executed by the processor.
[0069] In addition, to achieve the above-mentioned purpose, the present invention also proposes a coding device, which includes: a processor, a memory, and an image decoding program and / or an image coding program stored in the memory and run on the processor, wherein the image decoding program implements the steps of the image decoding method described above when executed by the processor, and the image coding program implements the steps of the image coding method described above when executed by the processor.
[0070] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which an image decoding program and / or an image encoding program are stored, and when the image decoding program is executed, the steps of the image decoding method as described above are implemented, and when the image encoding program is executed, the steps of the image encoding method as described above are implemented.
[0071] In addition, to achieve the above objectives, the present invention also proposes a computer program, which is configured to, when executed by a processor having a memory, implement the steps of the image decoding method as described above, or implement the steps of the image encoding method as described above.
[0072] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer program product, including computer program instructions, which are configured to, when executed by a processor with a memory, implement the steps of the image decoding method as described above, or implement the steps of the image encoding method as described above.
[0073] The present invention extracts image residual data or extended residual data from an image code stream, obtains multiple extended residual groups based on the extracted image residual data or the extended residual data; performs residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group; performs spatial domain resolution enlargement processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data; performs image reconstruction according to the reconstruction feature data to obtain a reconstructed image block. Since the extended residual data obtained is residual data processed with reduced spatial domain resolution, residual recovery processing can be performed in groups at low resolution, thereby improving the overall residual recovery calculation efficiency and reducing time complexity. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] Figure 1 It is a schematic diagram of the structure of an electronic device in a hardware operating environment involved in an embodiment of the present invention;
[0075] Figure 2 A schematic diagram of a flow chart of a first embodiment of an image decoding method of the present invention;
[0076] Figure 3 This is an overall framework diagram of image compression according to an embodiment of the present invention;
[0077] Figure 4 A schematic diagram of a flow chart of a second embodiment of an image decoding method according to the present invention;
[0078] Figure 5 A schematic diagram of a spatial resolution processing flow according to an embodiment of the present invention
[0079] Figure 6 A schematic diagram of a flow chart of a third embodiment of an image decoding method according to the present invention;
[0080] Figure 7 A schematic diagram of an image decoding group execution flow according to an embodiment of the present invention;
[0081] Figure 8 A schematic diagram of a secondary grouping execution process according to an embodiment of the present invention;
[0082] Fig. 9 A schematic diagram of a feature enhancement grouping execution process according to an embodiment of the present invention;
[0083] Fig.10 It is a flowchart of a first embodiment of an image encoding method of the present invention;
[0084] Fig.11 A schematic diagram of a flow chart of a second embodiment of an image encoding method according to the present invention;
[0085] Fig.12 is a structural block diagram of a first embodiment of an image decoding device according to the present invention;
[0086] Fig.13 FIG. 4 is a structural block diagram of the first embodiment of the image encoding device of the present invention.
[0087] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0088] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0089] Reference Figure 1 , Figure 1 The present invention is a schematic diagram of the structure of a decoding device or encoding device in a hardware operating environment involved in an embodiment of the present invention.
[0090] like Figure 1As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0091] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0092] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image decoding program and / or an image encoding program.
[0093] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in a decoding device or an encoding device, and the electronic device calls the image decoding program or the image encoding program stored in the memory 1005 through the processor 1001, and executes the image decoding method or the image encoding method provided by the embodiment of the present invention.
[0094] The embodiment of the present invention provides an image decoding method, referring to Figure 2 , Figure 2 The figure is a flowchart of a first embodiment of an image decoding method according to the present invention.
[0095] In this embodiment, the image decoding method includes the following steps:
[0096] Step S10: extracting image residual data or extended residual data from the image code stream, and obtaining a plurality of extended residual groups based on the extracted image residual data or the extended residual data.
[0097] It should be noted that the executor of this embodiment can be a decoding device when decoding image data. The decoding device can be an electronic device such as a personal computer or a server. Of course, it can also be other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image decoding method of the present invention is explained by taking a decoding device as an example.
[0098] Among them, since during the image encoding process, the encoding device generally decodes the encoded image code stream after the encoding is completed, and determines whether the parameters used in the encoding need to be adjusted according to the image quality of the decoded image, the executor of this embodiment can also be the encoding device.
[0099] It should be noted that the image code stream can be a code stream generated by the encoding device after encoding the image data to be compressed and encoded. When generating the image code stream, the encoding device reduces the spatial resolution of the image features to reduce the time complexity of predicting the mean during the encoding process. Finally, the encoding device will directly encode the generated image residual data or extended residual data into the image code stream. At this time, the decoding device can directly extract the image residual data or extended residual data from the image code stream. At this time, the decoding device can process the extracted image residual data or extended residual data to obtain multiple extended residual groups, and then perform residual recovery in groups, thereby reducing the time complexity of mean prediction in the image decoding process.
[0100] The technical terms involved in encoding or decoding images include: JPEG (Joint Photographic Experts Group), JPEG-AI (Joint Photographic Experts Group Artificial Intelligence), entropy coding (Entropy Encoding), neural network (Neural Network, NN), convolutional neural network (Convolutional Neural Network, CNN), feature (feature), rate-distortion principle (Rate-Distortion Optimized), etc., which are explained here.
[0101] Among them, JPEG (Joint Photographic Experts Group) is a standard for continuous tone static image compression. The file suffix is .jpg or .jpeg, and it is the most commonly used image file format. It mainly adopts a joint coding method of predictive coding (for example, Differential Pulse Code Modulation, DPCM), discrete cosine transform (Discrete Cosine Transform, DCT) and entropy coding to remove redundant image and color data. It is a lossy compression format. It can compress the image in a very small storage space, which will cause damage to the image data to a certain extent. In particular, using too high a compression ratio will reduce the quality of the image restored after the final decompression. If you pursue high-quality images, it is not advisable to use too high a compression ratio.
[0102] The scope of JPEG-AI is to create a learning-based image coding standard that provides a single-stream, compact compressed domain representation that is targeted for human visualization, significantly improves compression efficiency over commonly used image coding standards at equivalent subjective quality, and effective performance for image processing and computer vision tasks. JPEG-AI targets a wide range of applications, such as cloud storage, visual surveillance, autonomous vehicles and devices, image acquisition storage and management, real-time monitoring of visual data, and media distribution. The goal is to design a coding solution that needs to significantly improve the compression efficiency of commonly used coding standards at the same subjective quality, and provide efficient compressed domain processing for machine learning-based image processing and computer vision tasks. Other key requirements include hardware / software implementation-friendly encoding and decoding, support for 8-bit and 10-bit depths, efficient encoding of images with text and graphics, and progressive decoding.
[0103] Entropy coding is a coding process that does not lose any information according to the entropy principle. Information entropy is the average amount of information in the source (a measure of uncertainty). Common entropy coding methods include Shannon coding, Huffman coding, and arithmetic coding.
[0104] The neural network in this application refers to an artificial neural network, not a biological neural network. A neural network is a computational model consisting of a large number of nodes (or neurons) connected to each other. In an artificial neural network, a neuron processing unit can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in the network are divided into three categories: input units, output units, and hidden units. The input unit receives signals and data from the outside world; the output unit realizes the output of the system processing results; the hidden unit is a unit between the input and output units that cannot be observed from outside the system. The connection weights between neurons reflect the connection strength between units, and the representation and processing of information are reflected in the connection relationship of the network processing units. Artificial neural network is a non-programmed, brain-like information processing method. Its essence is to obtain a parallel distributed information processing function through the transformation and dynamic behavior of the network, and imitate the information processing function of the human brain nervous system to varying degrees and levels. At present, in the field of video processing, commonly used neural networks include convolutional neural networks (CNN), recurrent neural networks (RNN), fully connected networks, etc.
[0105] Convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. Its artificial neurons can respond to surrounding units within a certain coverage area and have excellent performance in large-scale image processing. Generally, the basic structure of CNN includes two layers. One is the feature extraction layer (also called convolution layer). The input of each neuron is connected to the local receptive field of the previous layer and extracts the local features. Once the local feature is extracted, its positional relationship with other features is also determined; the second is the feature mapping layer (also called activation layer). Each computing layer of the network consists of multiple feature maps. Each feature map is a plane, and the weights of all neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since neurons on a mapping surface share weights, the number of free parameters of the network is reduced. One of the advantages of CNN compared to traditional image processing algorithms is that it avoids the complex pre-processing process of the image (extracting artificial features, etc.), and can directly input the original image for end-to-end learning. One of the advantages of CNN over traditional neural networks is that traditional neural networks are fully connected, that is, all neurons from the input layer to the hidden layer are connected. This will result in a huge number of parameters, making network training time-consuming or even difficult to train. CNN avoids this difficulty through local connections, weight sharing and other methods.
[0106] The feature involved in this application is a three-dimensional feature matrix of CxWxH (such as Figure 3 As shown, Figure 3 is a schematic diagram of the matrix structure of this embodiment). C represents the number of channels, H represents the feature height, and W represents the feature width. The feature matrix can be the input of the neural network or the output of the neural network.
[0107] The indicators for evaluating coding efficiency include: bit rate, PSNR, MS-SSIM, VMAF FSIM, PSNRHVS and other indicators. Of course, there are more indicators, which are not limited here. The smaller the bit stream, the greater the compression rate; the larger the PSNR, the better the image coding efficiency. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. The cost corresponding to the mode: J(mode) = D + λ*R. Among them, D represents Distortion, which is usually measured by the SSE indicator. SSE refers to the mean square sum of the differences between the reconstructed block and the source image; λ is the Lagrange multiplier; R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, residuals, etc.
[0108] In a possible implementation of this embodiment, the obtained extended residual data are grouped according to the feature channels corresponding to each extended residual data. In this case, step S10 of this embodiment may include:
[0109] Extract extended residual data from the image code stream;
[0110] The extended residual data are grouped based on feature channels corresponding to the extended residual data to obtain a plurality of extended residual groups.
[0111] It should be noted that, the extended residual data is grouped based on the characteristic channels corresponding to the extended residual data to obtain multiple extended residual groups, which can be obtained by evenly dividing the extended residual data into multiple groups according to the corresponding characteristic channels. For example, assuming that the total number of characteristic channels corresponding to the extended residual data is 20, the extended residual data corresponding to the characteristic channels 1-10 can be grouped into one group, and the extended residual data corresponding to the characteristic channels 11-20 can be grouped into one group. The number of evenly divided groups can be pre-set by the administrator of the decoding device, and this embodiment does not limit this.
[0112] Of course, the specific grouping can also be unevenly divided. In this case, the extended residual data can be grouped based on the characteristic channels corresponding to the extended residual data to obtain multiple extended residual groups. The extended residual data can also be divided into multiple groups based on the corresponding characteristic channels based on preset grouping rules, wherein the preset grouping rules can be pre-set by the administrator of the decoding device according to actual needs. For example: the preset grouping rules are set to group the first m / n (n is the total number of characteristic channels, m is a preset value, and the value range is [1, n)) extended residual data into one group, and the remaining extended residual data into another group.
[0113] In actual use, when the extended residual data are grouped based on the feature channels corresponding to the extended residual data to obtain multiple extended residual groups, the extended residual data corresponding to one feature channel can also be divided into one group. For example, assuming that the total number of feature channels corresponding to the extended residual data is 20, the extended residual data can be divided into 20 groups according to the different feature channels.
[0114] Step S20: performing residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group.
[0115] It should be noted that performing residual recovery on the extended residual group to obtain the image reconstruction features corresponding to the extended residual group can be to perform mean prediction on the extended residual group, and then add the means obtained by predicting the residual data domain in the extended residual group to obtain the image reconstruction features corresponding to the extended residual group.
[0116] Step S30: performing spatial domain resolution enlargement processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data.
[0117] It should be noted that since the extended residual data has been processed with a reduced spatial resolution, the spatial size and the number of channels corresponding to each data are different from the spatial size and the number of channels of the image features obtained by the initial encoding device to extract features from the original image. At this time, in order to ensure the smooth execution of image reconstruction, the image reconstruction features corresponding to each extended residual group can be enlarged in spatial resolution, and the spatial size and the number of channels corresponding to the image reconstruction features can be restored to be consistent with the image features obtained by extracting features from the original image.
[0118] The process of enlarging the spatial resolution may be the reverse process of the process of reducing the spatial resolution in the encoding device.
[0119] Step S40: performing image reconstruction according to the reconstruction feature data to obtain a reconstructed image block.
[0120] It should be noted that after obtaining the reconstruction feature data whose spatial size and number of channels are consistent with the image features corresponding to the original image, the image can be reconstructed according to the reconstruction feature data to obtain a reconstructed image block.
[0121] The image is reconstructed according to the reconstructed feature data to obtain the reconstructed image block by performing a synthetic transformation process on the reconstructed feature data through a pre-constructed synthetic transformation network to achieve image reconstruction, thereby obtaining the reconstructed image block. The synthetic transformation network may be a network constructed based on deep learning or a neural network.
[0122] For ease of understanding, now combined Figure 3 This is for illustration only, but not for limitation. Figure 3 This is the overall framework diagram of image compression, such as Figure 3 As shown, on the encoding device side: x is the input image, and the main encoder (Analysis TransformNet) generates the potential representation y. Directly encoding y requires a large bit rate, so the context (Context ModelNet) and hyperparameter encoding network (Hyper Encoder net) are introduced, and the hyperparameter decoding network (Hyper Decoder net) is used for prediction to obtain the prediction result μ, and the residual resi = y-μ, and the distribution parameter σ of the residual is obtained through the hyperparameter encoding network (Hyper Encoder net) and the probability hyperparameter decoding network (Hyper Scale Decoder net). The distribution parameter can help entropy coding to encode data with less bit rate. Therefore, at the decoding end, it is necessary to obtain the predicted mean μ and distribution parameter σ of each y in order to correctly perform entropy decoding. The resi and distribution parameter σ are scaled by the G-unit component to control the quantization loss. The quantized residual obtained by rounding resi is recorded as Finally, lossless entropy encoding (Lossless encoder) is performed.
[0123] On the decoding device side: entropy decoding is performed on the image bitstream to obtain invG-unit component pair Scaling is performed, where the scaling multiples of the G-unit and invG-unit modules are opposite, and after scaling, combined with the predicted value μ, we get The synthesis transform net (Synthesis Transform Net) input to the decoding end obtains the reconstructed image block.
[0124] This embodiment extracts image residual data or extended residual data from the image code stream, obtains multiple extended residual groups based on the extracted image residual data or extended residual data; performs residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group; performs spatial resolution enlargement processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data; performs image reconstruction based on the reconstruction feature data to obtain a reconstructed image block. Since the extended residual data obtained is residual data that has been processed with reduced spatial resolution, residual recovery processing can be performed in groups at low resolution, thereby improving the overall residual recovery calculation efficiency and reducing time complexity.
[0125] refer to Figure 4 , Figure 4 The figure is a flowchart of a second embodiment of an image decoding method according to the present invention.
[0126] Based on the first embodiment, step S10 of the image decoding method of this embodiment includes:
[0127] Step S101: extracting image residual data from an image code stream.
[0128] It should be noted that the encoding device may expand the spatial resolution of the generated extended residual data, restore it to image residual data with a spatial size and number of channels consistent with the image features corresponding to the original image, and then encode the image residual data into the image bitstream. At this time, the decoding device can only extract the image residual data from the image bitstream when decoding the image bitstream.
[0129] Step S102: reducing the spatial resolution of the image residual data to obtain extended residual data.
[0130] It is understandable that, in order to facilitate subsequent grouping processing to reduce time complexity, after obtaining the image residual data, the image residual data can be processed by reducing the spatial resolution to obtain the expanded residual data. In order to ensure the correctness of decoding, the encoding device and the decoding device need to use the same method when performing the spatial resolution reduction process, and the spatial resolution enlargement process is the inverse process of the spatial resolution reduction process.
[0131] Among them, after the image residual data is processed by reducing the spatial resolution, its spatial size will become smaller. Correspondingly, when processing the image residual data, a smaller convolution kernel can be used. For example, if a convolution kernel with a size of 5x5 is used on the original image residual data, in terms of the receptive field of the convolution kernel, it is equivalent to using a convolution kernel with a size of 3x3 after reducing the spatial resolution.
[0132] In actual use, the step of reducing the spatial resolution of the image residual data to obtain the extended residual data in this embodiment may include:
[0133] The spatial size corresponding to the image residual data is reduced, and / or the number of feature channels corresponding to the image residual data is increased to obtain extended residual data.
[0134] It should be noted that the reduction range of the spatial size corresponding to the image residual data and the increase range of the number of feature channels corresponding to the image residual data can be pre-set by the administrator of the decoding device, and this embodiment does not limit this.
[0135] Among them, during actual execution, only the reduction range of the spatial size can be set, or only the increase range of the number of feature channels can be set, and the decoding device can adaptively adjust the spatial size or the number of feature channels. Of course, it is also possible not to use device adaptation, but to set the reduction range of the spatial size and the increase range of the number of feature channels, and let the device execute.
[0136] For example: assuming that the image feature corresponding to the image residual data is y∈R{H, W, C}, where H is the height of the image feature, W is the width of the image feature, and C is the number of feature channels corresponding to the image feature. At this time, H and W can be reduced to half of the original. In order to ensure that the amount of data remains unchanged, the number of feature channels will become 4 times the original. The image feature corresponding to the extended residual data obtained at this time can be expressed as y∈R{H / 2, W / 2, 4C}.
[0137] In a specific implementation, the spatial resolution reduction process may be performed based on the spatial information or frequency information corresponding to the image residual data, and the spatial resolution reduction process may also be performed through a preset convolution layer. At this time, the step of reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain the extended residual data described in this embodiment may include:
[0138] Reducing the spatial size corresponding to the image residual data according to the spatial domain information corresponding to the image residual data, and / or increasing the number of feature channels corresponding to the image residual data, so as to obtain extended residual data;
[0139] or,
[0140] Reducing the spatial size corresponding to the image residual data according to the frequency domain information corresponding to the image residual data, and / or increasing the number of feature channels corresponding to the image residual data, so as to obtain extended residual data;
[0141] or,
[0142] The spatial size corresponding to the image residual data is reduced according to a preset convolutional layer, and / or the number of feature channels corresponding to the image residual data is increased to obtain extended residual data.
[0143] For ease of understanding, now combined Figure 5 This is for illustration only, but not for limitation. Figure 5 Schematic diagram of the spatial resolution processing flow of this embodiment. Figure 5 As shown in method (a) in the figure, SpaceShuffle is one of the methods for reducing the spatial resolution. Assuming that the input of the processing process is a1∈R{H, W, C} and the output is a2∈R{H / 2, W / 2, 4*C}, the mathematical expression is as follows:
[0144] a2[h, w, c] = a1 [2*h, 2*w, c]
[0145] a2[h,w,C+c]=a1[2*h+1,2*w+1,c]
[0146] a2[h,w,2*C+c]=a1[2*h,2*w+1,c]
[0147] a2[h,w,3*C+c]=a1[2*h+1,2*w,c]
[0148] Among them, h represents the index value in the height dimension, and its value range is 0 to H-1, w represents the index value in the width dimension, and its value range is 0 to w-1, and c represents the index value in the channel dimension, and its value range is 0 to C-1. And a1[h, w, c] represents the size of the corresponding h, w, c index position in a1. Figure 5 In method (a), unSpaceShuffle is the inverse process of SpaceShuffle. UnSpaceShuffle is one of the methods for enlarging spatial resolution. In this process, after the input is processed by SpaceShuffle and unSpaceShuffle, the output result is the same as the original input, so this process is lossless. The input and output of the above processing process can also be other data, which is not limited in the embodiments of the present disclosure.
[0149] As shown in method (b), method (b) is another method for processing spatial resolution based on the spatial information corresponding to the image residual data. The specific sampling method of Pixshuffle is similar to that of method (a), but the difference is that the data is interleaved in the channel dimension. The data of the previous channel will be divided into four channels according to the spatial domain and arranged in sequence.
[0150] Figure 5Method (c) in the figure performs spatial resolution processing based on the frequency domain information corresponding to the image residual data. As shown in method (c), Wavelet transform is a two-dimensional wavelet transform that can divide the data in the frequency domain and output four frequency domain subbands with dimensions of [H / 2, W / 2, C], each of which represents different frequency domain characteristics. Inv Wavelet transform is its inverse process, which synthesizes the frequency domain subbands into the original data. This process is lossless.
[0151] Figure 5 Method (d) in the figure is to process the spatial resolution based on the preset convolution layer. As shown in method (d), Convolution is a convolution transformation, which is directly input into the convolution layer for processing to obtain the output of [H / 2, W / 2, 4*C], thereby reducing the spare resolution. The corresponding inverse process is also processed by the convolution layer to restore the original image size, but this method is lossy, and the data after both processing is different from the source data.
[0152] In a possible implementation of this embodiment, before the image residual data is processed to reduce the spatial resolution, the image residual data may be grouped first, and then the spatial resolution is reduced respectively. In this case, step S102 of this embodiment includes:
[0153] Grouping data according to feature channels corresponding to the image residual data to obtain at least one image residual group;
[0154] The data in the image residual group is processed to reduce the spatial resolution to obtain extended residual data.
[0155] It should be noted that the same or similar method as that of grouping the extended residual data can be used when grouping data according to the feature channels corresponding to the image residual data. The data in the image residual group is processed by reducing the spatial resolution to obtain the extended residual data, which can be performed by reducing the spatial resolution of the data in at least one image residual group, and then aggregating the data after the spatial resolution reduction process to obtain the extended residual data.
[0156] For example: assuming that the image feature corresponding to the image residual data is y∈R{H, W, C}, it can be divided into two groups according to the feature channel. Then the image feature corresponding to the data in each image residual group can be expressed as y∈R{H, W, C / 2}.
[0157] Step S103: grouping the extended residual data to obtain a plurality of extended residual groups.
[0158] It should be noted that grouping the extended residual data to obtain a plurality of extended residual groups can be performed by evenly dividing the extended residual data into a plurality of groups according to the corresponding characteristic channels. For example, assuming that the total number of characteristic channels corresponding to the extended residual data is 20, then the extended residual data corresponding to characteristic channels 1-10 can be grouped into one group, and the extended residual data corresponding to characteristic channels 11-20 can be grouped into one group. The number of evenly divided groups can be pre-set by the administrator of the decoding device, and this embodiment does not limit this.
[0159] Of course, the specific grouping can also be unevenly divided. The extended residual data can be grouped based on the characteristic channels corresponding to the extended residual data to obtain multiple extended residual groups. The extended residual data can also be divided into multiple groups based on the corresponding characteristic channels based on preset grouping rules, where the preset grouping rules can be pre-set by the administrator of the decoding device according to actual needs. For example: the preset grouping rules are set to group the first m / n (n is the total number of characteristic channels, m is a preset value, and the value range is [1, n)) extended residual data into one group, and the remaining extended residual data into another group.
[0160] In actual use, the extended residual data are grouped based on the feature channels corresponding to the extended residual data. When multiple extended residual groups are obtained, the extended residual data corresponding to one feature channel can also be divided into one group. For example, assuming that the total number of feature channels corresponding to the extended residual data is 20, the extended residual data can be divided into 20 groups according to the different feature channels.
[0161] This embodiment will also detect whether the extracted data is image residual data or extended residual data before grouping. If it is image residual data, it will first be processed by reducing the spatial resolution to ensure that even if the image code stream transmitted by the encoding end contains image residual data, it can still be grouped and processed normally after processing, thereby improving the versatility of the image decoding method of this embodiment.
[0162] refer to Figure 6 , Figure 6 FIG. 4 is a flow chart of a third embodiment of an image decoding method according to the present invention.
[0163] Based on the above first embodiment, step S20 of the image decoding method of this embodiment includes:
[0164] Step S201: constructing a residual recovery sequence according to the multiple extended residual groups.
[0165] It should be noted that after dividing into multiple extended residual groups, residual recovery can be performed in groups, thereby reducing the time complexity of mean prediction in the image decoding process. At this time, in order to determine the residual recovery order of each extended residual group, a residual recovery sequence can be constructed according to the multiple extended residual groups.
[0166] Step S202: performing residual recovery on the multiple extended residual groups respectively based on the residual recovery sequence to obtain image reconstruction features corresponding to each extended residual group.
[0167] It should be noted that performing residual recovery on multiple extended residual groups based on the residual recovery sequence may be performing residual recovery on multiple extended residual groups in sequence based on a sequence order in the residual recovery sequence.
[0168] In actual use, residual recovery may be performed sequentially by sequence traversal. In this case, step S202 of this embodiment may include:
[0169] Traversing the residual recovery sequence to obtain a current extended residual group;
[0170] Obtain auxiliary information output by the auxiliary encoding network;
[0171] constructing prior information based on the auxiliary information;
[0172] Performing residual recovery on the current extended residual group based on the prior information to obtain image reconstruction features corresponding to the current extended residual group;
[0173] At the end of the traversal, the image reconstruction features corresponding to each extended residual group are obtained.
[0174] It should be noted that traversing the residual recovery sequence to obtain the current extended residual group may be traversing the residual recovery sequence and taking the traversed extended residual group as the current extended residual group. Figure 3 The auxiliary network (Hyper Encoder Net or Hyper Decoder Net) shown in .
[0175] In actual use, residual recovery is performed on the current extended residual group based on prior information to obtain image reconstruction features corresponding to the current extended residual group. The prior information can be processed through a prediction parameter fusion network (Prediction Fusion Net) to obtain a predicted mean, and then the predicted mean is added to the residual in the current extended residual group to achieve residual recovery and obtain image reconstruction features corresponding to the current extended residual group.
[0176] In actual use, the step of constructing the prior information according to the auxiliary information in this embodiment may include:
[0177] Get extended auxiliary information;
[0178] Detecting whether the current extended residual group is the first element in the residual recovery sequence;
[0179] If it is the first element, constructing the prior information according to the extended auxiliary information;
[0180] If it is not the first element, the extended auxiliary information is spliced with the convolution processing result corresponding to the image reconstruction feature of the restored extended residual group to obtain spliced auxiliary information, and prior information is constructed according to the spliced auxiliary information.
[0181] It should be noted that the spatial size and number of feature channels corresponding to the auxiliary information output by the auxiliary coding network are actually consistent with those of the original image, and the extended residual data has actually been processed by reducing the spatial resolution. Therefore, in order to ensure smooth channel stitching, it is necessary to process the auxiliary information first to obtain the extended auxiliary information, and then construct the prior information based on the extended auxiliary information.
[0182] When constructing prior information based on the extended auxiliary information, in order to increase the accuracy of the mean prediction, when constructing the prior information, the image features corresponding to the previously reconstructed extended residual group can also be combined to construct the prior information. If the current extended residual group is the first element in the residual recovery sequence, it means that the extended residual group is the first extended residual group to perform residual recovery. At this time, there is no reconstructed extended residual group. Therefore, the prior information can be directly constructed based on the extended auxiliary information.
[0183] If the current extended residual group is not the first element in the residual recovery sequence, then there is an extended residual group that has been reconstructed. Therefore, the image reconstruction features of the restored extended residual group can be convolved through the convolution layer, and then the extended auxiliary information and the convolution processing results corresponding to the image reconstruction features of the restored extended residual group are channel-joined, and then the prior information is constructed based on the stitched auxiliary information. When constructing the prior information and selecting the restored extended residual group, all the restored extended residual groups can be selected, or only a part of the restored extended residual groups can be selected.
[0184] In a possible implementation of this embodiment, when performing result splicing, the image reconstruction features of the restored extended residual group may be enhanced to further improve the prediction effect. The step of splicing the extended auxiliary information with the convolution processing results corresponding to the image reconstruction features of the restored extended residual group to obtain the splicing auxiliary information may include:
[0185] Obtaining image reconstruction features corresponding to the restored extended residual group;
[0186] Performing feature enhancement on the image reconstruction feature to obtain an enhanced reconstruction feature;
[0187] The auxiliary information is spliced with the convolution processing result corresponding to the enhanced reconstruction feature to obtain spliced auxiliary information.
[0188] In actual use, the image reconstruction features are enhanced to obtain the enhanced reconstruction features by performing operations such as missing value processing and outlier processing on the image reconstruction features.
[0189] It can be understood that before splicing the auxiliary information with the image reconstruction features, the image reconstruction features are first enhanced to obtain enhanced reconstruction features, which can increase the reliability of the enhanced reconstruction features, thereby improving the reliability of the prior information, making the mean prediction based on the prior information more accurate.
[0190] In actual use, when the image reconstruction features corresponding to the restored extended residual group are enhanced, the predicted mean, auxiliary information, image residual data and / or residual data variance corresponding to the restored extended residual group may be used. At this time, the step of enhancing the image reconstruction features to obtain enhanced reconstruction features described in this embodiment may include:
[0191] Obtaining a predicted mean, auxiliary information, image residual data and / or residual data variance corresponding to the restored extended residual group;
[0192] The image reconstruction feature is enhanced according to the predicted mean, auxiliary information, image residual data and / or residual data variance corresponding to the restored extended residual group to obtain an enhanced reconstruction feature.
[0193] It should be noted that the predicted mean corresponding to the restored extended residual group may be a value obtained by performing mean prediction when performing residual recovery on the restored extended residual group. The residual data variance may be a variance value of the image residual data corresponding to the restored extended residual group.
[0194] In a possible implementation of this embodiment, in order to improve the reconstruction effect of image reconstruction, step S30 in this embodiment may include:
[0195] Performing feature enhancement on the image reconstruction features corresponding to each extended residual group to obtain enhanced reconstruction features corresponding to each extended residual group;
[0196] The spatial resolution of the enhanced reconstruction features corresponding to each extended residual group is enlarged to obtain the reconstructed feature data.
[0197] It should be noted that, the image reconstruction features corresponding to the extended residual group are enhanced to obtain the enhanced reconstruction features corresponding to the extended residual group, which can be achieved by using the predicted mean, auxiliary information, image residual data and / or residual data variance corresponding to the extended residual group to enhance the image reconstruction features corresponding to the extended residual group.
[0198] It can be understood that before the image reconstruction features corresponding to each extended residual group are enlarged in spatial domain resolution to obtain the reconstructed feature data, the image reconstruction features corresponding to each extended residual group are first enhanced, and then the enhanced reconstruction features corresponding to each extended residual group are enlarged in spatial domain resolution to obtain the reconstructed feature data. This can ensure that the reliability of the reconstructed feature data finally constructed is higher, and the quality of the reconstructed image blocks obtained by subsequent image reconstruction will be better.
[0199] For ease of understanding, now combined Figure 7 , 8 and 9 are used to illustrate, but not to limit this solution. Figure 7 FIG. 1 is a schematic diagram of the image decoding group execution flow of this embodiment. Figure 8 Schematic diagram of the secondary grouping execution process of this embodiment. Fig. 9 Schematic diagram of the feature enhancement grouping execution process of this embodiment.
[0200] like Figure 7 As shown in the figure, the image feature corresponding to the image residual data is y∈R{H, W, C}. First, the spatial resolution is reduced, and the result is y∈R{H / 2, W / 2, 4C}. At this time, it can be divided into two groups (group1 and group2), and then the residual recovery sequence (group1-group2) is constructed. Then, the mean mu of Group1 points is obtained through the network through the auxiliary (Psi) information, and the image reconstruction features of Group1 points are obtained. The image reconstruction features of Group1 points are used to extract features, and the channels are spliced with Psi. The mean mu of Group2 is obtained through the network, and then decoded to obtain Group2. Combine Group1 points and Group2 for channel splicing, and then perform special processing to enlarge the spatial resolution (that is, the inverse process of reducing the spatial resolution) to obtain reconstructed feature data, and then reconstruct the image based on the reconstructed feature data.
[0201] If grouping is performed before reducing the spatial resolution, the execution flow is as follows: Figure 8As shown, the image feature corresponding to the image residual data is y∈R{H, W, C}. At this time, it is evenly divided into y1 and y2 according to the corresponding feature channel, so y1∈R{H, W, C / 2} is obtained. At this time, the spatial resolution is reduced and y1 is divided into part1, part2, part3, and part4. At this time, the following exists:
[0202] Part1 is represented by the even-numbered rows and even-numbered columns of the spatial index of y1 in y1, and contains data information of all channels. Part2 is represented by the odd-numbered rows and odd-numbered columns of the spatial index of y1 in y1, and contains data information of all channels. Part3 is represented by the even-numbered rows and odd-numbered columns of the spatial index of y1 in y1, and contains data information of all channels. Part4 is represented by the odd-numbered rows and even-numbered columns of the spatial index of y1 in y1, and contains data information of all channels.
[0203] Similarly, y2 can also be divided into four similar parts, namely part5, part6, part7, and part8. Then the residual recovery sequence constructed at this time is "part1-part2-part3-part4-part5-part6-part7-part8". Then, when performing residual recovery on part1, prior information can be directly generated based on the auxiliary information. When performing residual recovery on part2, prior information can be generated based on the image reconstruction features and auxiliary information of part1. When performing residual recovery on part3, prior information can be generated based on the image reconstruction features and auxiliary information of parts1 and 2, and so on, until the image reconstruction features corresponding to all parts are obtained;
[0204] Of course, two residual recovery sequences can also be constructed according to y1 and y2 respectively. In this case, the two residual recovery sequences are "part1-part2-part3-part4" and "part5-part6-part7-part8" respectively. At this time, a process similar to the above can be used to reconstruct the image of y1 according to the residual recovery sequence composed of part1-4, and obtain the image reconstruction features corresponding to each part in y1. Then, prior information is generated according to the image reconstruction features corresponding to y1 and auxiliary information. Then, image y2 is reconstructed according to the residual recovery sequence composed of part5-8 and the generated prior information, and image reconstruction features corresponding to each part in y2 are obtained.
[0205] Among them, since y is first split into y1 and y2 and then the spatial resolution is reduced respectively, after obtaining the image reconstruction features corresponding to each part in y1 and y2, the image reconstruction features corresponding to each part of y1 and y2 can be enlarged in the spatial resolution and then aggregated to obtain complete reconstructed feature data.
[0206] If feature enhancement is performed after grouping, the specific execution process is as follows: Fig. 9 As shown in (a), the image feature corresponding to the image residual data is y∈R{H, W, C}. First, the spatial resolution is reduced, and the result is y∈R{H / 2, W / 2, 4C}. At this time, it can be divided into two groups (group1 and group2). Then, the mean mu of the Group1 points is obtained through the network through the auxiliary (Psi) information, and the image reconstruction feature of Group1 is obtained. The image reconstruction feature of Group1 is enhanced using the predicted mean corresponding to Group1 to obtain the enhanced image feature Group1_E. Then, the features of the enhanced image feature are extracted, and the channel is spliced with Psi. The mean mu of Group2 is obtained through the network, and then the image reconstruction feature of Group2 is decoded. Then, the image reconstruction feature of Group2 is enhanced according to the mean mu of Group2 to obtain the enhanced image feature Group2_E of Group2. Group1_E and Group2_E are spliced through the channels, and then a special process of enlarging the spatial resolution is performed (that is, the inverse process of reducing the spatial resolution) to obtain the reconstructed feature data, and then the image is reconstructed based on the reconstructed feature data. Among them, Fig. 9 When feature enhancement is performed in (a), the specific structure of the network (Enhance_Net) used for feature enhancement is as follows Fig. 9 As shown in (b).
[0207] In this embodiment, a residual recovery sequence is constructed according to the multiple extended residual groups; based on the residual recovery sequence, residual recovery is performed on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group. Since the residual recovery sequence is constructed according to the multiple extended residual groups, the order of residual recovery can be determined by the residual recovery sequence, so that it is possible to quickly determine whether there is a restored extended residual group, and when there is a restored extended residual group, more accurate prior information is constructed according to the image feature data corresponding to the restored extended residual group.
[0208] The embodiment of the present invention provides an image encoding method, referring to Fig.10 , Fig.10 FIG. 4 is a flow chart of a first embodiment of an image encoding method according to the present invention.
[0209] In this embodiment, the image encoding method includes the following steps:
[0210] Step S100: reducing the spatial resolution of image features corresponding to the image to be encoded to obtain extended image features.
[0211] It should be noted that, in order to facilitate subsequent grouping processing and reduce time complexity, after obtaining the image features corresponding to the image to be encoded, the image residual data can be processed by reducing the spatial resolution to obtain extended image features. The image to be encoded is the original image mentioned in the above-mentioned image decoding method embodiment.
[0212] Among them, after the image features are processed by reducing the spatial resolution, their spatial size will become smaller. Correspondingly, when processing the image features, a smaller convolution kernel can be used. For example, if a convolution kernel with a size of 5x5 is used on the original image residual data, in terms of the receptive field of the convolution kernel, it is equivalent to using a convolution kernel with a size of 3x3 after reducing the spatial resolution.
[0213] In a possible implementation of this embodiment, step S100 in this embodiment may include:
[0214] The spatial size corresponding to the image feature of the image to be encoded is reduced, and / or the number of feature channels corresponding to the image feature is increased to obtain the extended image feature.
[0215] It should be noted that the reduction range of the spatial size corresponding to the image feature and the increase range of the number of feature channels corresponding to the image feature can be pre-set by the administrator of the decoding device, and this embodiment does not limit this.
[0216] Among them, during actual execution, only the reduction range of the spatial size can be set, or only the increase range of the number of feature channels can be set, and the decoding device can adaptively adjust the spatial size or the number of feature channels. Of course, it is also possible not to use device adaptation, but to set the reduction range of the spatial size and the increase range of the number of feature channels, and let the device execute.
[0217] In a specific implementation, the spatial resolution reduction process may be performed based on the spatial information or frequency information corresponding to the image feature, and the spatial resolution reduction process may also be performed through a preset convolution layer. In this case, the step of reducing the spatial size corresponding to the image feature of the image to be encoded and / or increasing the number of feature channels corresponding to the image feature to obtain the extended image feature described in this embodiment may include:
[0218] Reducing the spatial size corresponding to the image feature of the image to be encoded according to the spatial domain information corresponding to the image feature, and / or increasing the number of feature channels corresponding to the image feature, so as to obtain an extended image feature;
[0219] or,
[0220] Reducing the spatial size corresponding to the image feature of the image to be encoded according to the frequency domain information corresponding to the image feature, and / or increasing the number of feature channels corresponding to the image feature, so as to obtain an extended image feature;
[0221] or,
[0222] The spatial size corresponding to the image feature is reduced according to a preset convolutional layer, and / or the number of feature channels corresponding to the image feature is increased to obtain the extended image feature.
[0223] The specific implementation method can refer to the above-mentioned image decoding method embodiment. Figure 5 The explanation part will not be repeated here.
[0224] Step S200: grouping the extended image features to obtain a plurality of extended feature groups.
[0225] In a specific implementation, when grouping the extended image features, the feature channels corresponding to the extended image features may be referred to for grouping. In this case, step S200 in this embodiment may include:
[0226] The extended image features are grouped based on feature channels corresponding to the extended feature data to obtain a plurality of extended feature groups.
[0227] It should be noted that, the extended image features are grouped based on the feature channels corresponding to the extended feature data, and the multiple extended feature groups can be obtained by evenly dividing the extended image features into multiple groups according to the corresponding feature channels. For example, assuming that the total number of feature channels corresponding to the extended image features is 20, then the extended image features with corresponding feature channels of 1-10 can be grouped into one group, and the extended image features with corresponding feature channels of 11-20 can be grouped into one group. The number of evenly divided groups can be pre-set by the administrator of the decoding device, and this embodiment does not limit this.
[0228] Of course, the specific grouping can also be unevenly divided. The extended image features can be grouped based on the feature channels corresponding to the extended image features to obtain multiple extended feature groups. The extended image features can also be divided into multiple groups based on the corresponding feature channels based on preset grouping rules. The preset grouping rules can be pre-set by the management personnel of the encoding device according to actual needs. For example: the preset grouping rules are set to group the first m / n (n is the total number of feature channels, m is a preset value, and the value range is [1, n)) extended image features into one group, and the remaining extended image features into another group.
[0229] In actual use, the extended image features are grouped based on the feature channels corresponding to the extended image features. When multiple extended feature groups are obtained, the extended image features corresponding to one feature channel can also be divided into one group. For example, assuming that the total number of feature channels corresponding to the extended image features is 20, the extended image features can be divided into 20 groups according to the different feature channels.
[0230] Step S300: performing residual calculations on the plurality of extended feature groups respectively to obtain image residual data corresponding to each extended feature group.
[0231] It should be noted that residual calculation is performed on the extended feature groups respectively to obtain image residual data corresponding to the extended feature groups. The extended feature groups can be predicted by mean, and then the image features in the extended feature groups are subtracted from the predicted mean to obtain image residual data corresponding to the extended feature groups.
[0232] Step S400: Generate an image code stream according to the image residual data, and send the image code stream to an image decoding end.
[0233] It should be noted that generating the image code stream according to the image residual data may be writing the image residual data into the image code stream through entropy coding.
[0234] In a possible implementation of this embodiment, before performing spatial domain resolution reduction processing on image features corresponding to the image to be encoded, the image features may be grouped first, and then spatial domain resolution reduction processing may be performed separately. In this case, step S100 of this embodiment may include:
[0235] Obtain image features corresponding to the image to be encoded;
[0236] Grouping the image features according to feature channels corresponding to the image features to obtain at least one image feature group;
[0237] The data in the image feature group is processed to reduce the spatial resolution to obtain extended image features.
[0238] In a possible implementation of this embodiment, the image features corresponding to the image to be encoded are processed by reducing the spatial resolution to obtain the extended image features, or after obtaining the image features corresponding to the image to be encoded, the data in the image features are processed by reducing the spatial resolution to obtain the extended image features. It should be noted that obtaining the image features corresponding to the image to be encoded may be extracting the image features corresponding to the image to be encoded through a preset feature extraction network. When grouping data according to the feature channels corresponding to the feature channels corresponding to the image features, the same or similar method as that for grouping the extended image features may be adopted.
[0239] In a possible implementation of this embodiment, step S400 described in this embodiment may include:
[0240] Performing spatial domain resolution processing on the image residual features corresponding to each extended feature group to obtain image residual data corresponding to the image to be encoded;
[0241] An image code stream is generated according to the image residual data corresponding to the image to be encoded, and the image code stream is sent to an image decoding end.
[0242] It should be noted that the processing of enlarging the spatial resolution can be the inverse process of the above-mentioned processing of reducing the spatial resolution. After obtaining the image residual features corresponding to each extended feature group, the image residual features corresponding to each extended feature group can be enlarged by spatial resolution, and the corresponding spatial size and number of channels can be restored to be consistent with the image features corresponding to the image to be encoded, so as to obtain the image residual data corresponding to the image to be encoded, and then the image residual data corresponding to the image to be encoded is entropy encoded to generate an image code stream, and the generated image code stream is sent to the image decoding end.
[0243] This embodiment obtains extended image features by reducing the spatial resolution of the image features corresponding to the image to be encoded; groups the extended image features to obtain multiple extended feature groups; performs residual calculations on the multiple extended feature groups to obtain image residual data corresponding to each extended feature group; generates an image code stream based on the image residual data, and sends the image code stream to the image decoding end. Since the image features of the image to be encoded are reduced in spatial resolution after being obtained, and then grouped into multiple extended feature groups, the residual calculation can be performed in groups at low resolution, thereby improving the overall residual calculation efficiency and reducing time complexity.
[0244] refer to Fig.11 , Fig.11 FIG. 4 is a flow chart of a second embodiment of an image encoding method according to the present invention.
[0245] Based on the above first embodiment, step S300 of the image encoding method of this embodiment includes:
[0246] Step S3001: constructing a residual calculation sequence according to the multiple extended feature groups.
[0247] It should be noted that after the extended feature groups are divided, residual calculation can be performed on the entire group. At this time, in order to determine the residual calculation order of each extended feature group, a residual calculation sequence can be constructed according to the multiple extended feature groups.
[0248] Step S3002: performing residual calculations on the plurality of extended feature groups respectively based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group.
[0249] It should be noted that performing residual calculations on the multiple extended feature groups based on the residual calculation sequence may be performing residual calculations on the multiple extended feature groups in sequence based on the sequence order in the residual calculation sequence.
[0250] In actual use, the residual calculation may be performed sequentially by sequence traversal. In this case, step S3002 of this embodiment may include:
[0251] Traversing the residual calculation sequence to obtain a current extended feature group;
[0252] Obtain auxiliary information output by the auxiliary encoding network;
[0253] constructing prior information based on the auxiliary information;
[0254] Performing residual calculation on the current extended feature group based on the prior information to obtain image residual data corresponding to the current extended feature group;
[0255] At the end of the traversal, the image residual data corresponding to each extended feature group is obtained.
[0256] It should be noted that traversing the residual calculation sequence to obtain the current extended feature group may be traversing the residual calculation sequence and taking the traversed extended feature group as the current extended feature group. Figure 3 The auxiliary network (Hyper Encoder Net or Hyper Decoder Net) shown in .
[0257] In actual use, residual calculation is performed on the current extended feature group based on prior information to obtain image residual data corresponding to the current extended feature group. The prior information can be processed by Prediction Fusion Net to obtain the predicted mean, and then the features in the current extended feature group are subtracted from the predicted mean to realize residual calculation and obtain image residual data corresponding to the current extended feature group.
[0258] Among them, when constructing prior information based on auxiliary information, in order to increase the accuracy of mean prediction, when constructing the prior information, the image features corresponding to the extended feature group whose residual calculation has been performed before can also be combined to construct the prior information. The specific implementation method is the same as the method used in the image decoding process, and the specific implementation steps can refer to the method of constructing prior information based on auxiliary information provided in any of the above-mentioned image decoding method embodiments.
[0259] In this embodiment, a residual calculation sequence is constructed according to multiple extended feature groups; residual calculations are performed on the multiple extended feature groups based on the residual calculation sequence to obtain image residual data corresponding to each extended feature group. Since the residual calculation sequence is constructed according to multiple extended feature groups, the order of residual calculations can be determined by the residual calculation sequence, so that it is possible to quickly determine whether there is an extended feature group that has been calculated, and when there is an extended feature group that has been calculated, more accurate prior information is constructed according to the convolution processing results corresponding to the image reconstruction features of the restored extended residual group.
[0260] In addition, an embodiment of the present invention also proposes a storage medium, on which an image decoding and / or image encoding program is stored. When the image decoding program is executed by a processor, the steps of the image decoding method described above are implemented; when the image encoding program is executed by a processor, the steps of the image encoding method described above are implemented.
[0261] Reference Fig.12 , Fig.12 This is a structural block diagram of the first embodiment of the image decoding device of the present invention.
[0262] like Fig.12 As shown, the image decoding device proposed in the embodiment of the present invention includes:
[0263] A code stream decoding module 10, configured to extract image residual data or extended residual data from an image code stream, and obtain a plurality of extended residual groups based on the extracted image residual data or the extended residual data;
[0264] A residual recovery module 20, configured to perform residual recovery on the plurality of extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group;
[0265] A data combination module 30 is used to enlarge the spatial resolution of the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data;
[0266] The image reconstruction module 40 is used to perform image reconstruction according to the reconstruction feature data to obtain a reconstructed image block.
[0267] This embodiment obtains multiple extended residual groups based on the image residual data or extended residual data extracted from the image code stream; performs residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group; performs spatial resolution enlargement processing on the image reconstruction features corresponding to each extended residual group to obtain reconstruction feature data; performs image reconstruction based on the reconstruction feature data to obtain a reconstructed image block. Since the obtained extended residual data is residual data that has been processed with reduced spatial resolution, residual recovery processing can be performed in groups at low resolution, thereby improving the overall residual recovery calculation efficiency and reducing time complexity.
[0268] In a possible implementation of this embodiment, the code stream decoding module 10 is further used to extract extended residual data from the image code stream; and group the extended residual data based on feature channels corresponding to the extended residual data to obtain multiple extended residual groups.
[0269] In a possible implementation of this embodiment, the bitstream decoding module 10 is further used to extract image residual data from the image bitstream; perform spatial resolution reduction processing on the image residual data to obtain extended residual data, and the spatial resolution enlargement processing is the inverse process of the spatial resolution reduction processing; and group the extended residual data to obtain multiple extended residual groups.
[0270] In a possible implementation of this embodiment, the bitstream decoding module 10 is further used to group data according to the feature channels corresponding to the image residual data to obtain at least one image residual group; and perform spatial resolution reduction processing on the data in the image residual group to obtain extended residual data.
[0271] In a possible implementation of this embodiment, the code stream decoding module 10 is further used to reduce the spatial size corresponding to the image residual data and / or increase the number of feature channels corresponding to the image residual data to obtain extended residual data.
[0272] In a possible implementation of the present embodiment, the bitstream decoding module 10 is further used to reduce the spatial size corresponding to the image residual data according to the spatial domain information corresponding to the image residual data, and / or increase the number of feature channels corresponding to the image residual data, so as to obtain extended residual data; or, reduce the spatial size corresponding to the image residual data according to the frequency domain information corresponding to the image residual data, and / or increase the number of feature channels corresponding to the image residual data, so as to obtain extended residual data; or, reduce the spatial size corresponding to the image residual data according to a preset convolutional layer, and / or increase the number of feature channels corresponding to the image residual data, so as to obtain extended residual data.
[0273] In a possible implementation of this embodiment, the residual recovery module 20 is further used to construct a residual recovery sequence according to the multiple extended residual groups; based on the residual recovery sequence, the multiple extended residual groups are respectively subjected to residual recovery to obtain image reconstruction features corresponding to each extended residual group.
[0274] In a possible implementation of the present embodiment, the residual recovery module 20 is further used to traverse the residual recovery sequence to obtain a current extended residual group; obtain auxiliary information output by the auxiliary coding network; construct prior information based on the auxiliary information; perform residual recovery on the current extended residual group based on the prior information to obtain image reconstruction features corresponding to the current extended residual group; and at the end of the traversal, obtain image reconstruction features corresponding to each extended residual group.
[0275] In a possible implementation of the present embodiment, the residual recovery module 20 is also used to obtain extended auxiliary information; detect whether the current extended residual group is the first element in the residual recovery sequence; if it is the first element, construct prior information based on the extended auxiliary information; if it is not the first element, splicing the extended auxiliary information with the convolution processing result corresponding to the image reconstruction features of the restored extended residual group to obtain spliced auxiliary information, and construct prior information based on the spliced auxiliary information.
[0276] In a possible implementation of this embodiment, the residual recovery module 20 is also used to obtain image reconstruction features corresponding to the restored extended residual group; perform feature enhancement on the image reconstruction features to obtain enhanced reconstruction features; and splice the auxiliary information with the convolution processing results corresponding to the enhanced reconstruction features to obtain spliced auxiliary information.
[0277] In a possible implementation of the present embodiment, the residual recovery module 20 is also used to obtain the predicted mean, auxiliary information, image residual data and / or residual data variance corresponding to the restored extended residual group; and perform feature enhancement on the image reconstruction features according to the predicted mean, auxiliary information, image residual data and / or residual data variance corresponding to the restored extended residual group to obtain enhanced reconstruction features.
[0278] In a possible implementation of the present embodiment, the data combination module 30 is further used to perform feature enhancement on the image reconstruction features corresponding to each extended residual group to obtain enhanced reconstruction features corresponding to each extended residual group; and to amplify the spatial resolution of the enhanced reconstruction features corresponding to each extended residual group to obtain reconstructed feature data.
[0279] In a possible implementation of the present embodiment, the image reconstruction module 40 is further used to perform synthetic transformation processing on the reconstructed feature data through a pre-constructed synthetic transformation network to realize the image reconstruction and obtain the reconstructed image block, wherein the synthetic transformation network is a network constructed based on deep learning or a neural network.
[0280] In a possible implementation manner of this embodiment, the code stream decoding module 10 is further configured to evenly group the extended residual data according to the feature channels corresponding to the extended residual data.
[0281] Reference Fig.13 , Fig.13 FIG. 4 is a structural block diagram of the first embodiment of the image encoding device of the present invention.
[0282] like Fig.13 As shown, the image encoding device proposed in the embodiment of the present invention includes:
[0283] The feature extraction module 100 is used to reduce the spatial resolution of the image features corresponding to the image to be encoded to obtain extended image features;
[0284] A data grouping module 200, configured to group the extended image features to obtain a plurality of extended feature groups;
[0285] The residual calculation module 300 is used to perform residual calculation on the multiple extended feature groups respectively to obtain image residual data corresponding to each extended feature group;
[0286] The code stream generation module 400 is used to generate an image code stream according to the image residual data, and send the image code stream to an image decoding end.
[0287] This embodiment obtains extended image features by reducing the spatial resolution of the image features corresponding to the image to be encoded; groups the extended image features to obtain multiple extended feature groups; performs residual calculations on the multiple extended feature groups to obtain image residual data corresponding to each extended feature group; generates an image code stream based on the image residual data, and sends the image code stream to the image decoding end. Since the image features of the image to be encoded are reduced in spatial resolution after being obtained, and then grouped into multiple extended feature groups, the residual calculation can be performed in groups at low resolution, thereby improving the overall residual calculation efficiency and reducing time complexity.
[0288] In a possible implementation of this embodiment, the feature extraction module 100 is further used to obtain image features corresponding to the image to be encoded; and to reduce the spatial resolution of the data in the image features to obtain extended image features.
[0289] In a possible implementation of this embodiment, the feature extraction module 100 is also used to
[0290] The spatial size corresponding to the image feature of the image to be encoded is reduced, and / or the number of feature channels corresponding to the image feature is increased to obtain the extended image feature.
[0291] In a possible implementation of the present embodiment, the feature extraction module 100 is further used to reduce the spatial size corresponding to the image feature according to the spatial domain information corresponding to the image feature of the image to be encoded, and / or increase the number of feature channels corresponding to the image feature to obtain an extended image feature; or, to reduce the spatial size corresponding to the image feature according to the frequency domain information corresponding to the image feature of the image to be encoded, and / or increase the number of feature channels corresponding to the image feature to obtain an extended image feature; or, to reduce the spatial size corresponding to the image feature according to a preset convolutional layer, and / or increase the number of feature channels corresponding to the image feature to obtain an extended image feature.
[0292] In a possible implementation of this embodiment, the residual calculation module 300 is further used to construct a residual calculation sequence according to the multiple extended feature groups; based on the residual calculation sequence, the multiple extended feature groups are respectively subjected to residual calculation to obtain image residual data corresponding to each extended feature group.
[0293] In a possible implementation of this embodiment, the residual calculation module 300 is also used to traverse the residual calculation sequence to obtain the current extended feature group; obtain auxiliary information output by the auxiliary coding network; construct prior information based on the auxiliary information; perform residual calculation on the current extended feature group based on the prior information to obtain image residual data corresponding to the current extended feature group; at the end of the traversal, obtain image residual data corresponding to each extended feature group.
[0294] In a possible implementation of this embodiment, the data grouping module 200 is further configured to group the extended image features based on feature channels corresponding to the extended feature data to obtain a plurality of extended feature groups.
[0295] In a possible implementation of this embodiment, the code stream generation module 400 is further used to perform a spatial resolution enlargement process on the image residual features corresponding to each extended feature group to obtain image residual data corresponding to the image to be encoded, and the spatial resolution enlargement process is the inverse process of the spatial resolution reduction process; generate an image code stream according to the image residual data corresponding to the image to be encoded, and send the image code stream to the image decoding end.
[0296] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.
[0297] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.
[0298] In addition, for technical details not fully described in this embodiment, reference may be made to the image decoding method or image encoding method provided in any embodiment of the present invention, and will not be repeated here.
[0299] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0300] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0301] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0302] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. An image decoding method, characterized in that: The image decoding method comprises: Extracting image residual data from an image code stream; performing spatial domain resolution reduction processing on the image residual data to obtain extended residual data; grouping the extended residual data to obtain a plurality of extended residual groups; Performing residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group; Performing a spatial resolution enlargement process on the image reconstruction features corresponding to each extended residual group to obtain reconstructed feature data, wherein the spatial resolution enlargement process is the inverse process of the spatial resolution reduction process; Image reconstruction is performed according to the reconstruction feature data to obtain a reconstructed image block.
2. The image decoding method according to claim 1, characterized in that: The step of reducing the spatial resolution of the image residual data to obtain extended residual data includes: The spatial size corresponding to the image residual data is reduced, and / or the number of feature channels corresponding to the image residual data is increased to obtain the extended residual data.
3. The image decoding method according to claim 2, characterized in that: The reducing the spatial size corresponding to the image residual data and / or increasing the number of feature channels corresponding to the image residual data to obtain the extended residual data includes: reducing the spatial size corresponding to the image residual data according to the spatial domain information corresponding to the image residual data to obtain the extended residual data; or, Increasing the number of feature channels corresponding to the image residual data according to the spatial domain information corresponding to the image residual data to obtain the extended residual data; or, The spatial size corresponding to the image residual data is reduced according to the spatial domain information corresponding to the image residual data, and the number of feature channels corresponding to the image residual data is increased according to the spatial domain information corresponding to the image residual data to obtain the extended residual data.
4. The image decoding method according to claim 1, characterized in that: The step of grouping the extended residual data to obtain a plurality of extended residual groups includes: The extended residual data are evenly grouped according to the feature channels corresponding to the extended residual data to obtain a plurality of extended residual groups.
5. The image decoding method according to any one of claims 1 to 4, characterized in that: When the image reconstruction features corresponding to each extended residual group are processed to enlarge the spatial resolution, The enlarged spatial resolution processing is used to restore the spatial size corresponding to the image reconstruction feature to be consistent with the spatial size corresponding to the image obtained by feature extraction of the original image, and the enlarged spatial resolution processing is used to restore the number of channels corresponding to the high image reconstruction feature to be consistent with the number of channels corresponding to the image obtained by feature extraction of the original image.
6. An image encoding method, characterized in that: The image encoding method comprises: Performing spatial domain resolution reduction processing on image features corresponding to the image to be encoded to obtain extended image features; Grouping the extended image features to obtain a plurality of extended feature groups; Residual calculations are performed on the multiple extended feature groups respectively to obtain image residual data corresponding to each extended feature group; wherein, after obtaining the image reconstruction features corresponding to each extended feature group, the image reconstruction features corresponding to each extended feature group are enlarged in spatial domain resolution processing to obtain image residual data corresponding to each extended feature group, and the enlarged in spatial domain resolution processing is the inverse process of the reduced in spatial domain resolution processing; An image code stream is generated according to the image residual data, and the image code stream is sent to an image decoding end.
7. An image decoding device, characterized in that: The image decoding device comprises: A code stream decoding module is used to extract image residual data from the image code stream; reduce the spatial resolution of the image residual data to obtain extended residual data; group the extended residual data to obtain multiple extended residual groups; A residual recovery module, used to perform residual recovery on the multiple extended residual groups respectively to obtain image reconstruction features corresponding to each extended residual group; A data combination module is used to perform a spatial resolution enlargement process on the image reconstruction features corresponding to each extended residual group to obtain reconstructed feature data; wherein the spatial resolution enlargement process is the inverse process of the spatial resolution reduction process; The image reconstruction module is used to perform image reconstruction according to the reconstruction feature data to obtain a reconstructed image block.
8. An image encoding device, characterized in that The image encoding device comprises: A feature extraction module is used to reduce the spatial resolution of the image features corresponding to the image to be encoded to obtain extended image features; A data grouping module, used for grouping the extended image features to obtain a plurality of extended feature groups; A residual calculation module is used to perform residual calculation on the multiple extended feature groups respectively to obtain image residual data corresponding to each extended feature group; wherein, after obtaining the image reconstruction features corresponding to each extended feature group, the residual calculation module performs a magnification spatial resolution process on the image reconstruction features corresponding to each extended feature group to obtain image residual data corresponding to each extended feature group, wherein the amplification spatial resolution process is the inverse process of the reduction spatial resolution process; The code stream generation module is used to generate an image code stream according to the image residual data, and send the image code stream to the image decoding end.
9. A decoding device, characterized in that: The decoding device comprises: a processor, a memory, and an image decoding program stored in the memory and executable on the processor, wherein the image decoding program implements the image decoding method according to any one of claims 1 to 5 when executed by the processor.
10. A coding device, characterized in that: The encoding device comprises: a processor, a memory, and an image encoding program stored in the memory and executable on the processor, wherein the image encoding program implements the image encoding method according to claim 6 when executed by the processor.
11. A storage medium, characterized in that: The storage medium stores an image decoding program and / or an image encoding program. When the image decoding program is executed, the image decoding method according to any one of claims 1 to 5 is implemented. When the image encoding program is executed, the image encoding method according to claim 6 is implemented.
12. A computer program product, characterized in that The computer program comprises computer program instructions, which are configured to, when executed by a processor having a memory, implement the image decoding method according to any one of claims 1 to 5, or implement the image encoding method according to claim 6.
Citation Information
Patent Citations
Out-chip buffer compression method for superhigh-definition processing system
CN105472389A
Super-resolution reconstruction method based on loop training
CN108765297A
Inter prediction method and apparatus
CN113556567A
Image reconstruction method, image coding and decoding method and related equipment
CN114943643A
Video data compaction coding method
CN1921627A
