Image decoding and coding method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of image processing, and discloses an image decoding and coding method and device, equipment and a storage medium. The method comprises the following steps: decoding an image code stream, and determining a feature reconstruction value corresponding to a current image feature obtained by decoding; performing feature enhancement on the feature reconstruction value to obtain an enhanced feature value; and performing synthetic transformation on the enhanced feature value to obtain a reconstructed image block. Before image reconstruction, feature enhancement is carried out on the feature reconstruction value, and then image reconstruction is carried out according to the enhanced feature enhancement value, so that distortion of image features in processes of quantization and the like is reduced, and the image quality of the reconstructed image is improved.

Description

[0001] This invention patent application is a divisional application of the Chinese invention patent application with application date of January 13, 2023, application number 202310055970.0, and name “Image decoding and encoding method, device, equipment and storage medium”. Technical Field

[0002] The present invention relates to the field of image processing technology, and in particular to an image decoding and encoding method, device, equipment and storage medium. Background Art

[0003] Nowadays, deep learning and neural networks continue to make breakthroughs in the field of video image compression. Image coding technology based on deep learning has already significantly surpassed traditional coding standards in coding performance. However, how to improve the image quality of images reconstructed after compression using image coding technology based on deep learning is a major challenge.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide an image decoding and encoding method, device, equipment and storage medium, aiming to improve the technical problem of image quality of images reconstructed after compression using deep learning-based image coding technology.

[0006] To achieve the above object, the present invention provides an image decoding method, which comprises the following steps:

[0007] Decoding the image code stream and determining a feature reconstruction value corresponding to the current image feature obtained by decoding;

[0008] Performing feature enhancement on the feature reconstruction value to obtain an enhanced feature value;

[0009] The enhanced feature values ​​are synthetically transformed to obtain a reconstructed image block.

[0010] In a possible implementation manner of the present application, the current image feature is a three-dimensional feature matrix;

[0011] The step of performing feature enhancement on the feature reconstruction value to obtain an enhanced feature value includes:

[0012] Obtaining the characteristic standard deviation and standard deviation representation value corresponding to each matrix element in the current image feature;

[0013] Determine a feature mask corresponding to each matrix element in the current image feature according to the feature standard deviation, the standard deviation representation value and a preset threshold value;

[0014] Based on the feature mask, feature enhancement is performed on the feature reconstruction value to obtain an enhanced feature value.

[0015] In a possible implementation manner of the present application, determining the feature mask corresponding to each matrix element in the current image feature according to the feature standard deviation, the standard deviation representation value and a preset threshold value includes:

[0016] If the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition, the characteristic mask corresponding to the matrix element is set to the first type value;

[0017] If the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element do not meet the preset enhancement condition, the characteristic mask corresponding to the matrix element is set to the second type value.

[0018] In a possible implementation manner of the present application, if the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition, before setting the characteristic mask corresponding to the matrix element to the first type value, it also includes:

[0019] If the characteristic standard deviation corresponding to the matrix element is greater than a preset threshold value, and the standard deviation characterization value is a first type characterization value, then it is determined that the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition;

[0020] or,

[0021] If the characteristic standard deviation corresponding to the matrix element is less than the preset definition threshold, and the standard deviation characterization value is a second type characterization value, it is determined that the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition.

[0022] In a possible implementation manner of the present application, the step of performing feature enhancement on the feature reconstruction value based on the feature mask to obtain an enhanced feature value includes:

[0023] The matrix element whose corresponding feature mask is a first type value is used as the target matrix element;

[0024] The eigenvalues ​​corresponding to the target matrix elements are enhanced to obtain enhanced eigenvalues.

[0025] In a possible implementation manner of the present application, the step of enhancing the eigenvalues ​​corresponding to the target matrix elements to obtain enhanced eigenvalues ​​includes:

[0026] Obtaining characteristic reconstruction values, residual reconstruction values ​​and predicted characteristic values ​​corresponding to the target matrix elements;

[0027] Determining a first enhancement value according to a first scaling factor and the residual reconstruction value, and determining a second enhancement value according to a second scaling factor and the predicted characteristic value;

[0028] An enhanced feature value is determined according to the feature reconstruction value, the first enhancement value, and the second enhancement value.

[0029] In a possible implementation manner of the present application, before determining the first enhancement value according to the first scaling factor and the predicted characteristic value, and determining the second enhancement value according to the second scaling factor and the residual reconstruction value, the method further includes:

[0030] Obtaining the feature channel corresponding to the target matrix element;

[0031] A first scaling factor and a second scaling factor are determined according to the characteristic channel, and different characteristic channels correspond to different first scaling factors and second scaling factors.

[0032] In a possible implementation manner of the present application, the step of enhancing the eigenvalues ​​corresponding to the target matrix elements to obtain enhanced eigenvalues ​​includes:

[0033] Obtaining the eigenreconstructed values ​​and predicted eigenvalues ​​corresponding to the target matrix elements;

[0034] Determining a first enhancement value according to a first scaling factor and the feature reconstruction value, and determining a second enhancement value according to a second scaling factor and the predicted feature value;

[0035] An enhancement feature value is determined according to the first enhancement value and the second enhancement value.

[0036] In a possible implementation manner of the present application, the step of enhancing the eigenvalues ​​corresponding to the target matrix elements to obtain enhanced eigenvalues ​​includes:

[0037] Obtaining characteristic reconstruction values ​​and residual reconstruction values ​​corresponding to the target matrix elements;

[0038] Determining a first enhancement value according to a first scaling factor and the feature reconstruction value, and determining a second enhancement value according to a second scaling factor and the residual reconstruction value;

[0039] An enhancement feature value is determined according to the first enhancement value and the second enhancement value.

[0040] In a possible implementation manner of the present application, the step of enhancing the eigenvalues ​​corresponding to the target matrix elements to obtain enhanced eigenvalues ​​includes:

[0041] Obtaining the characteristic reconstruction value and characteristic standard deviation corresponding to the target matrix element;

[0042] Determining a first enhancement value according to a first scaling factor and the feature reconstruction value, and determining a second enhancement value according to a second scaling factor and the feature standard deviation;

[0043] An enhancement feature value is determined according to the first enhancement value and the second enhancement value.

[0044] In a possible implementation manner of the present application, the feature reconstruction value includes a brightness reconstruction value and a chroma reconstruction value, and the enhanced feature value includes a brightness enhancement feature value and a chroma enhancement feature value;

[0045] The step of enhancing the eigenvalues ​​corresponding to the target matrix elements to obtain enhanced eigenvalues ​​includes:

[0046] Enhance the chroma reconstruction value corresponding to the target matrix element to obtain a chroma enhancement eigenvalue;

[0047] The brightness reconstruction value corresponding to the target matrix element is enhanced to obtain a brightness enhancement eigenvalue.

[0048] In a possible implementation manner of the present application, after enhancing the chroma reconstruction value corresponding to the target matrix element to obtain the chroma enhancement eigenvalue, the method further includes:

[0049] Determine a first enhancement value according to the brightness reconstruction value and a first scaling factor;

[0050] The chroma enhancement feature value is secondary enhanced according to the first enhancement value.

[0051] In a possible implementation manner of the present application, determining the first enhancement value according to the brightness reconstruction value and the first scaling factor further includes:

[0052] Extract component indication parameters from the image code stream;

[0053] If the component indication parameter is a chroma enhancement type parameter, a first enhancement value is determined according to the luma reconstruction value and a first scaling factor.

[0054] In a possible implementation manner of the present application, decoding the image code stream and determining a feature reconstruction value corresponding to a current image feature obtained by decoding includes:

[0055] Decoding the image code stream and determining a residual reconstruction value corresponding to the current image feature obtained by decoding;

[0056] Predicting based on the feature reconstruction value of the reconstructed feature to obtain a predicted feature value;

[0057] A feature reconstruction value corresponding to the current image feature is determined according to the residual reconstruction value and the predicted feature value.

[0058] In a possible implementation manner of the present application, after decoding the image code stream and determining the residual reconstruction value corresponding to the current image feature obtained by decoding, the method further includes:

[0059] Predicting is performed according to the enhanced feature value of the reconstructed feature to obtain a predicted feature value;

[0060] A feature reconstruction value corresponding to the current image feature is determined according to the residual reconstruction value and the predicted feature value.

[0061] In a possible implementation manner of the present application, the enhanced feature value of the reconstructed feature and the enhanced feature value corresponding to the current image feature are enhanced in the same manner or in a different manner.

[0062] In a possible implementation of the present application, whether the enhanced feature value of the reconstructed feature is the same as or different from the enhanced feature value corresponding to the current image feature is determined by a syntax flag, and the syntax flag is read from an image code stream.

[0063] In a possible implementation manner of the present application, the step of performing feature enhancement on the feature reconstruction value to obtain an enhanced feature value includes:

[0064] Extracting a syntax application interval parameter from an image code stream, and obtaining feature position information corresponding to the current image feature;

[0065] Determine enhanced grammar parameters according to the feature position information and the grammar application interval parameters;

[0066] The feature reconstruction value is feature enhanced according to the enhanced syntax parameter to obtain an enhanced feature value.

[0067] In addition, to achieve the above object, the present invention also provides an image decoding device, which includes the following modules:

[0068] A code stream decoding module, used to decode the image code stream and determine a feature reconstruction value corresponding to the current image feature obtained by decoding;

[0069] A feature enhancement module, used to perform feature enhancement on the feature reconstruction value to obtain an enhanced feature value;

[0070] The image reconstruction module is used to perform a synthetic transformation on the enhanced feature value to obtain a reconstructed image block.

[0071] In addition, to achieve the above object, the present invention also proposes an image encoding method, the image encoding method comprising:

[0072] Extract features from the image block to be encoded, and use the extracted features as current image features;

[0073] Predicting is performed according to the feature reconstruction value corresponding to the reconstructed feature to obtain the predicted feature value;

[0074] Determine the encoding residual coefficient corresponding to the current image feature according to the predicted feature value;

[0075] The coding residual coefficient is written into the image code stream corresponding to the image block to be coded.

[0076] In a possible implementation manner of the present application, after writing the coding residual coefficient into the image code stream corresponding to the image block to be coded, the method further includes:

[0077] Decoding the image code stream corresponding to the image block to be encoded to obtain a reconstructed image block;

[0078] The image coding efficiency is determined according to the reconstructed image block and the image block to be encoded.

[0079] In addition, to achieve the above object, the present invention further provides an image encoding device, the image encoding device comprising:

[0080] A feature extraction module, used for extracting features from the image block to be encoded, and using the extracted features as current image features;

[0081] A feature prediction module is used to predict according to the feature reconstruction value corresponding to the reconstructed feature to obtain a predicted feature value;

[0082] A residual calculation module, used to determine the encoding residual coefficient corresponding to the current image feature according to the predicted feature value;

[0083] The parameter writing module is used to write the coding residual coefficient into the image code stream corresponding to the image block to be encoded.

[0084] In addition, to achieve the above-mentioned purpose, the present invention also proposes a decoding device, which includes: a processor, a memory, and a decoding program stored in the memory and executable on the processor, and when the decoding program is executed by the processor, the image decoding method as described above is implemented.

[0085] In addition, to achieve the above-mentioned purpose, the present invention also proposes a coding device, which includes: a processor, a memory, and a decoding program and / or encoding program stored in the memory and run on the processor, wherein the decoding program implements the image decoding method described above when executed by the processor, and the encoding program implements the image encoding method described above when executed by the processor.

[0086] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which an image decoding program and / or an image encoding program are stored, and when the image decoding program is executed, the image decoding method as described above is implemented, and when the image encoding program is executed, the image encoding method as described above is implemented.

[0087] The present invention decodes the image code stream and determines the feature reconstruction value corresponding to the current image feature obtained by decoding; performs feature enhancement on the feature reconstruction value to obtain an enhanced feature value; and performs a synthetic transformation on the enhanced feature value to obtain a reconstructed image block. Because the feature reconstruction value is firstly feature enhanced before image reconstruction, and then the image is reconstructed according to the enhanced feature enhancement value, the distortion of the image feature in the process of quantization is reduced, thereby improving the image quality of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 It is a schematic diagram of the structure of an electronic device in a hardware operating environment involved in an embodiment of the present invention;

[0089] Figure 2 A schematic diagram of a flow chart of a first embodiment of an image decoding method of the present invention;

[0090] Figure 3 A schematic diagram of a matrix structure according to an embodiment of the present invention;

[0091] Figure 4 A schematic diagram of an image encoding and decoding process according to an embodiment of the present invention;

[0092] Figure 5 A schematic diagram of an image encoding and decoding process according to an embodiment of the present invention;

[0093] Figure 6 A schematic diagram of a flow chart of a second embodiment of an image decoding method according to the present invention;

[0094] Figure 7 A schematic diagram of a flow chart of a third embodiment of an image decoding method according to the present invention;

[0095] Figure 8 A schematic diagram of a flow chart of a fourth embodiment of an image decoding method of the present invention

[0096] Fig. 9 A schematic diagram of a feature reconstruction sequence according to an embodiment of the present invention;

[0097] Fig.10 It is a flowchart of a first embodiment of an image encoding method of the present invention;

[0098] Fig.11 A schematic diagram of a flow chart of a second embodiment of an image encoding method according to the present invention

[0099] Fig.12 is a structural block diagram of a first embodiment of an image decoding device according to the present invention;

[0100] Fig.13 FIG. 4 is a structural block diagram of the first embodiment of the image encoding device of the present invention.

[0101] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0102] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0103] Reference Figure 1 , Figure 1 A decoding device or a schematic diagram of the decoding device structure in a hardware operating environment involved in an embodiment of the present invention.

[0104] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0105] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0106] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a decoding and / or encoding program.

[0107] exist Figure 1In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in a decoding device or a decoding device, and the electronic device calls the decoding program and / or decoding program stored in the memory 1005 through the processor 1001, and executes the image decoding method or image encoding method provided by an embodiment of the present invention.

[0108] The embodiment of the present invention provides an image decoding method, referring to Figure 2 , Figure 2 The figure is a flowchart of a first embodiment of an image decoding method according to the present invention.

[0109] In this embodiment, the image decoding method includes the following steps:

[0110] Step S10: Decode the image code stream and determine a feature reconstruction value corresponding to the current image feature obtained by decoding.

[0111] It should be noted that the executor of this embodiment can be a decoding device when encoding image data. The decoding device can be an electronic device such as a personal computer or a server. Of course, it can also be other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image decoding method of the present invention is explained by taking a decoding device as an example.

[0112] Among them, since during the image encoding process, the encoding device generally decodes the encoded image code stream after the encoding is completed, and determines whether the parameters used in the encoding need to be adjusted according to the image quality of the decoded image, the executor of this embodiment can also be the encoding device.

[0113] It should be noted that the image code stream may be a code stream generated after the encoding device encodes the image data to be compressed and encoded. When the decoding device decodes the image code stream, it extracts the image features of the encoded image data from the image code stream, and the image features currently obtained during the decoding process are the current image features. The feature reconstruction value may be the image feature obtained after the feature recovery of the current image features during the decoding process.

[0114] In the specific processing process, when encoding the image data, the encoding device may divide the image data into one image block for processing. Of course, the image data may also be divided into multiple image blocks for processing, which is not limited in this embodiment.

[0115] The technical terms involved in encoding or decoding images include: JPEG (Joint Photographic Experts Group), JPEG-AI (Joint Photographic Experts Group Artificial Intelligence), entropy coding (Entropy Encoding), neural network (Neural Network, NN), convolutional neural network (Convolutional Neural Network, CNN), feature (feature), rate-distortion principle (Rate-Distortion Optimized), etc., which are explained here.

[0116] Among them, JPEG (Joint Photographic Experts Group) is a standard for continuous tone static image compression. The file suffix is ​​.jpg or .jpeg, and it is the most commonly used image file format. It mainly adopts a joint coding method of predictive coding (DPCM), discrete cosine transform (DCT) and entropy coding to remove redundant image and color data. It is a lossy compression format. It can compress images in a very small storage space, which will cause damage to image data to a certain extent. In particular, using too high a compression ratio will reduce the quality of the image restored after decompression. If you pursue high-quality images, it is not advisable to use too high a compression ratio.

[0117] The scope of JPEG AI is to create a learning-based image coding standard that provides a single-stream, compact compressed domain representation that targets both human visualization, significantly improves compression efficiency over commonly used image coding standards at equivalent subjective quality, and effective performance for image processing and computer vision tasks. JPEG AI targets a wide range of applications such as cloud storage, visual surveillance, autonomous vehicles and devices, image acquisition storage and management, real-time monitoring of visual data, and media distribution. The goal is to design a coding solution that needs to significantly improve the compression efficiency of commonly used coding standards at the same subjective quality, and provide efficient compressed domain processing for machine learning-based image processing and computer vision tasks. Other key requirements include hardware / software implementation-friendly encoding and decoding, support for 8-bit and 10-bit depths, efficient encoding of images with text and graphics, and progressive decoding.

[0118] Entropy coding is a coding process that does not lose any information according to the entropy principle. Information entropy is the average amount of information in the source (a measure of uncertainty). Common entropy coding methods include Shannon coding, Huffman coding, and arithmetic coding.

[0119] The neural network in this application refers to an artificial neural network, not a biological neural network. A neural network is a computational model consisting of a large number of nodes (or neurons) connected to each other. In an artificial neural network, a neuron processing unit can represent different objects, such as features, letters, concepts, or some meaningful abstract patterns. The types of processing units in the network are divided into three categories: input units, output units, and hidden units. The input unit receives signals and data from the outside world; the output unit realizes the output of the system processing results; the hidden unit is a unit between the input and output units that cannot be observed from outside the system. The connection weights between neurons reflect the connection strength between units, and the representation and processing of information are reflected in the connection relationship of the network processing units. Artificial neural network is a non-programmed, brain-like information processing method. Its essence is to obtain a parallel distributed information processing function through the transformation and dynamic behavior of the network, and imitate the information processing function of the human brain nervous system to varying degrees and levels. At present, in the field of video processing, commonly used neural networks include convolutional neural networks (CNN), recurrent neural networks (RNN), fully connected networks, etc.

[0120] Convolutional neural network is a feedforward neural network and one of the most representative network structures in deep learning technology. Its artificial neurons can respond to surrounding units within a certain coverage area and have excellent performance in large-scale image processing. Generally, the basic structure of CNN includes two layers. One is the feature extraction layer (also called convolution layer). The input of each neuron is connected to the local receptive field of the previous layer and extracts the local features. Once the local feature is extracted, its positional relationship with other features is also determined; the second is the feature mapping layer (also called activation layer). Each computing layer of the network consists of multiple feature maps. Each feature map is a plane, and the weights of all neurons on the plane are equal. The feature mapping structure can use Sigmoid function, ReLU function, Leaky-ReLU function, PReLU function, GDN function, etc. as the activation function of the convolutional network. In addition, since neurons on a mapping surface share weights, the number of free parameters of the network is reduced. One of the advantages of CNN compared to traditional image processing algorithms is that it avoids the complex pre-processing process of the image (extracting artificial features, etc.), and can directly input the original image for end-to-end learning. One of the advantages of CNN over traditional neural networks is that traditional neural networks are fully connected, that is, all neurons from the input layer to the hidden layer are connected. This will result in a huge number of parameters, making network training time-consuming or even difficult to train. CNN avoids this difficulty through local connections, weight sharing and other methods.

[0121] The feature involved in this application is a three-dimensional feature matrix of CxWxH (such as Figure 3 As shown, Figure 3 is a schematic diagram of the matrix structure of this embodiment). C represents the number of channels, H represents the feature height, and W represents the feature width. The feature matrix can be the input of the neural network or the output of the neural network.

[0122] There are two major indicators for evaluating coding efficiency: bit rate and PSNR. The smaller the bit stream, the greater the compression rate; the larger the PSNR, the better the image coding efficiency. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. The cost corresponding to the mode: J(mode) = D + λ*R. Among them, D represents Distortion, which is usually measured by the SSE indicator. SSE refers to the mean square sum of the differences between the reconstructed block and the source image; λ is the Lagrange multiplier; R is the actual number of bits required for encoding the image block under this mode, including the sum of bits required for coding mode information, motion information, residuals, etc. When selecting a mode, if the RDO principle is used to make comparative decisions on the coding mode, the best coding performance can usually be guaranteed.

[0123] In a possible implementation manner of the present application, the value of the image feature may be relatively large or even relatively complex. In order to improve the coding efficiency, when encoding the image feature, the coding device may perform residual calculation on the image feature and then encode the obtained residual data. When decoding the image data, a corresponding recovery process is required to ensure that a feature reconstruction value close to the image feature before encoding can be obtained. At this time, step S10 described in this embodiment may include:

[0124] Decoding the image code stream and determining a residual reconstruction value corresponding to the current image feature obtained by decoding;

[0125] Predicting based on the feature reconstruction value of the reconstructed feature to obtain a predicted feature value;

[0126] A feature reconstruction value corresponding to the current image feature is determined according to the residual reconstruction value and the predicted feature value.

[0127] It should be noted that if the encoding device calculates the residual data before encoding, then decoding the image code stream can only obtain the residual reconstruction value corresponding to the current image feature. The reconstructed feature can be a part of the image feature that has completed feature recovery, and prediction is performed based on the feature reconstruction value of the reconstructed feature, and obtaining the predicted feature value can be performed using a mean prediction network based on the feature reconstruction value of the reconstructed feature to obtain the predicted feature value.

[0128] For ease of understanding, now combined Figure 4 To explain, Figure 4FIG. 1 is a schematic diagram of the image encoding and decoding process of this embodiment. In the figure, Bitstream#1 is an auxiliary code stream, and Bitstream#2 is an image code stream. Figure 4 As shown, the feature of the original image block (i.e., the image data that the encoding device needs to encode) is extracted through the analysis transformation network to obtain the image feature y, and the auxiliary information z_hat is calculated through the hyperparameter encoding network, and then the predicted feature value mu is calculated through the mean prediction network. The encoding device will perform residual processing to obtain the residual original value r of the current feature, and then after residual processing and quantization (Q&AE), the obtained encoding residual coefficient r_coef is written into the image bitstream (Bitstream#2).

[0129] Afterwards, when the decoding device processes the image bitstream (Bitstream#2), it can extract the coding residual coefficient r_coef from the image bitstream, and then perform inverse quantization and residual recovery (AD&IQ) on the coding residual coefficient, and then obtain the residual reconstruction value r_hat corresponding to the current image feature, and then predict the feature reconstruction value y_hat of the reconstructed feature through the mean prediction network to obtain the predicted feature value mu, and determine the feature reconstruction value y_hat of the current image feature according to the predicted feature value and the residual reconstruction value, and then perform feature enhancement on the feature reconstruction value to obtain the enhanced feature value y_hat_en, and finally perform synthetic encoding on the enhanced feature value through the synthetic transformation network to obtain the reconstructed image block x_hat. Among them, the parameters used in the processing of Q&AE and AD&IQ are obtained by processing the auxiliary information z_hat through the probabilistic hyperparameter decoding network.

[0130] Among them, the analysis transformation network, hyperparameter encoding network, probabilistic hyperparameter decoding network and synthetic transformation network can all be neural networks built based on deep learning.

[0131] In actual use, the feature reconstruction value corresponding to the current image feature can be determined based on the residual reconstruction value and the predicted feature value by adding the residual reconstruction value and the predicted feature value, and using the sum obtained after the addition as the feature reconstruction value corresponding to the current image feature. When making a prediction, the feature reconstruction value of a reconstructed feature close to the position of the current image feature can be used for prediction.

[0132] In a possible implementation of the present application, in order to further improve the coding efficiency, the coding device may, in the process of encoding the image, after calculating the residual data, perform residual processing or quantization processing on the residual data, and then write the processed coding residual coefficient into the image code stream. At this time, determining the residual reconstruction value corresponding to the current image feature obtained by decoding can be to decode the image code stream, extract the coding residual coefficient corresponding to the current image feature, dequantize and residually recover the coding residual coefficient, and obtain the residual reconstruction value corresponding to the current image feature.

[0133] In a possible implementation manner of the present application, when making a prediction, the enhanced feature value of the reconstructed feature may also be used for prediction. In this case, step S10 in this embodiment may include:

[0134] Decoding the image code stream and determining a residual reconstruction value corresponding to the current image feature obtained by decoding;

[0135] Predicting is performed according to the enhanced feature value of the reconstructed feature to obtain a predicted feature value;

[0136] A feature reconstruction value corresponding to the current image feature is determined according to the residual reconstruction value and the predicted feature value.

[0137] It should be noted that in some cases, when the encoder is encoding the image data, it may use the enhanced feature value of the reconstructed feature for prediction when calculating the residual value, determine the predicted feature value corresponding to the currently encoded image feature, and calculate the encoded residual coefficient of the currently encoded image feature based on the predicted feature value. In the decoding process, the same method is also needed to predict according to the enhanced feature value of the reconstructed feature through the mean prediction network to obtain the predicted feature value, and then add the residual reconstruction value and the predicted feature value to obtain the feature reconstruction value corresponding to the current image feature.

[0138] For ease of understanding, now combined Figure 5 To explain, Figure 5 FIG. 1 is a schematic diagram of the image encoding and decoding process of this embodiment. Figure 5 As shown, during the encoding and decoding of image data, the processing flow is the same as above Figure 4Basically similar, but the difference is that during the decoding process, after obtaining the feature reconstruction value of the current image feature, the first feature enhancement method will be used to perform feature enhancement on the feature reconstruction value (i.e., feature enhancement 1 in the figure), and the obtained first enhanced feature value y_hat_en1 will be input into the synthesis transformation network for synthesis transformation processing to obtain the reconstructed image block x_hat. At the same time, the second feature enhancement method will be used to perform feature enhancement on the feature reconstruction value (i.e., feature enhancement 2 in the figure), and the obtained second enhanced feature value y_hat_en2 will be input into the mean prediction network. When other image features are processed thereafter, the current image feature will be used as the reconstructed feature, and the second enhanced feature value will be used for prediction to calculate the predicted feature value.

[0139] It should be noted that the purpose of feature enhancement 2 is to Figure 5 The subsequent feature prediction values ​​shown are generated one by one (columns can be diagonal columns, not necessarily vertical columns), and the features of the entire image (or the entire image block) cannot be completed at the same time. When feature enhancement 1 is performed, the features of the entire image (or the entire image block) are often reconstructed, so the features of the entire image (or the entire image block) can be executed in parallel.

[0140] In actual use, the feature enhancement methods of the enhanced feature value of the reconstructed feature and the enhanced feature value corresponding to the current image feature can be the same or different. That is, the feature enhancement process of the first feature enhancement method and the second feature enhancement method can be the same process, then y_hat_en1 is equal to y_hat_en2, and the decoding device only needs to decode a set of related syntax parameters from the image code stream. Of course, according to actual needs, the first feature enhancement method and the second feature enhancement method can also be set as different processes, then the decoding device needs to decode two sets of syntax parameters from the image code stream.

[0141] Among them, in specific applications, the administrator of the decoding device or the encoding device can also set configuration parameters. By setting the configuration parameters, it is allowed to skip the first feature enhancement method and / or the second feature enhancement method, that is, directly set y_hat_en1 and / or y_hat_en2 equal to y_hat. The configuration parameters can be set in the image code stream. Of course, you can also try to set the configuration parameters in other ways, and this embodiment is not limited to this.

[0142] In a possible implementation of the present application, the encoding device may pre-write a syntax flag in the image code stream to indicate whether the first feature enhancement method and the second feature enhancement method use exactly the same parameters. At this time, the decoding device may determine whether the first feature enhancement method and the second feature enhancement method use exactly the same parameters based on the syntax flag read from the image code stream. For example, the encoding device uses a 1-bit syntax flag useSameParaFlag to characterize whether the first feature enhancement method and the second feature enhancement method use exactly the same syntax parameters. If useSameParaFlag=1 read by the decoding device, it indicates that the first feature enhancement method and the second feature enhancement method use exactly the same syntax parameters. If useSameParaFlag=0 read by the decoding device, it indicates that the first feature enhancement method and the second feature enhancement method use completely different syntax parameters.

[0143] In a possible implementation of the present application, the grammatical parameters involved in the feature enhancement method may be shown in the following table:

[0144] Table 1 Syntax parameter semantics table

[0145]

[0146]

[0147] In a possible implementation of the present application, on the basis of the syntax parameters in Table 1, other parameters may be additionally added, such as: blockSizeList[idx] and modeList[idx], wherein the encoding length of blockSizeList[idx] may be 8 bits, and the semantics are: for a block size of NxN, if the value of N is 1, it means that it is based on each pixel. Specifically, if N is greater than 1, a representative value of the NxN block is obtained by taking the minimum value, maximum value, mean, etc., and the representative value is enhanced based on the representative value based on the feature enhancement method of the present invention to obtain a new representative value, and then the enhanced value of the NxN block is set to the new representative value; the encoding length of modeList[idx] may be 3 bits, and the semantics are: values ​​of 1 to 4 respectively indicate that min, avg, max, and max pool are used for up and down sampling (for block size not 1x1), and a value of 5 indicates that a set of filters has 2 scales, then the parameters are as shown in Table 2:

[0148] Table 2 Syntax parameter semantics table

[0149]

[0150]

[0151] Step S20: performing feature enhancement on the feature reconstruction value to obtain an enhanced feature value.

[0152] It should be noted that, since the image features will undergo quantization and other processing processes when encoding the image data, the features will be partially distorted. The feature reconstruction values ​​can be enhanced first, and then the image can be reconstructed based on the enhanced feature values ​​obtained by the enhancement, thereby improving the image quality of the reconstructed image.

[0153] Step S30: performing a synthetic transformation on the enhanced feature value to obtain a reconstructed image block.

[0154] It should be noted that the enhanced feature values ​​are synthetically encoded to obtain the reconstructed image block by performing synthetic transformation processing on the enhanced feature values ​​through a pre-constructed synthetic transformation network, thereby obtaining the reconstructed image block. The synthetic transformation network may be a network constructed based on deep learning or a neural network.

[0155] It is understandable that if the encoding device divides the image data into only one image block for processing during encoding, the reconstructed image block obtained at this time is the reconstructed image data corresponding to the image data. If the encoding device divides the image data into multiple image blocks for processing during encoding, the reconstructed image block obtained at this time is only the reconstructed image data corresponding to a certain image block in the image data.

[0156] This embodiment decodes the image code stream and determines the feature reconstruction value corresponding to the current image feature obtained by decoding; performs feature enhancement on the feature reconstruction value to obtain an enhanced feature value; and performs a synthetic transformation on the enhanced feature value to obtain a reconstructed image block. Because the feature reconstruction value is firstly feature enhanced before image reconstruction, and then the image is reconstructed according to the enhanced feature enhancement value, the distortion of the image feature in the process of quantization is reduced, thereby improving the image quality of the reconstructed image.

[0157] refer to Figure 6 , Figure 6 The figure is a flowchart of a second embodiment of an image decoding method according to the present invention.

[0158] Based on the above first embodiment, step S20 of the image decoding method of this embodiment includes:

[0159] Step S201: Obtain the feature standard deviation and standard deviation representation value corresponding to each matrix element in the current image feature.

[0160] It should be noted that the current image feature can be a three-dimensional feature matrix, and its three-dimensional dimensions can be represented by c, i, and j, where c is the channel identifier, and i and j respectively identify the feature height and feature width. The feature standard deviation corresponding to the matrix element can be the standard deviation between the feature corresponding to the matrix element and the feature mean, and the standard deviation characterization value is used to characterize whether the standard deviation corresponding to the matrix element is a large standard deviation.

[0161] Among them, the standard deviation characterization value can be divided into a first type characterization value and a second type characterization value (for the sake of simplicity, true and false can be used to represent it, true is the first type characterization value, and false is the second type characterization value). If the standard deviation characterization value corresponding to the matrix element is the first type characterization value, it means that the standard deviation corresponding to the matrix element is a large standard deviation; if the standard deviation characterization value corresponding to the matrix element is the second type characterization value, it means that the standard deviation corresponding to the matrix element is not a large standard deviation.

[0162] Step S202: determining a feature mask corresponding to each matrix element in the current image feature according to the feature standard deviation, the standard deviation representation value and a preset threshold.

[0163] It should be noted that the feature mask can be a representation value used to identify whether to perform feature enhancement on the feature reconstruction value corresponding to the matrix element, wherein the feature mask can be divided into a first type value and a second type value (for the sake of simplicity, true and false can be used to represent it, true is the first type value, and false is the second type value). If the feature mask corresponding to the matrix element is the first type value, it means that the feature reconstruction value corresponding to the matrix element needs to be enhanced; if the feature mask corresponding to the matrix element is the second type value, it means that the feature reconstruction value corresponding to the matrix element does not need to be enhanced.

[0164] In actual use, a preset enhancement condition may be preset to determine whether the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition to set the characteristic mask of the matrix element. In this case, step S202 of this embodiment may include:

[0165] If the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition, the characteristic mask corresponding to the matrix element is set to the first type value;

[0166] If the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element do not meet the preset enhancement condition, the characteristic mask corresponding to the matrix element is set to the second type value.

[0167] It can be understood that if the feature standard deviation and standard deviation characterization value corresponding to the matrix element meet the preset enhancement conditions, it means that the feature reconstruction value corresponding to the matrix element needs to be enhanced. Therefore, the feature mask corresponding to the matrix element can be set to the first type value.

[0168] If the feature standard deviation and standard deviation representation value corresponding to the matrix element do not meet the preset enhancement conditions, it means that the feature reconstruction value corresponding to the matrix element does not need to be enhanced. Therefore, the feature mask corresponding to the matrix element can be set to the second type value.

[0169] In a specific implementation, feature enhancement can be performed on matrix elements with large standard deviations and corresponding feature standard deviations greater than a certain threshold, and feature enhancement can be performed on matrix elements with non-large standard deviations and corresponding feature standard deviations less than a certain threshold. The preset threshold can be pre-set by a manager of an encoding device or a decoding device. In this case, if the feature standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition, the step of setting the feature mask corresponding to the matrix element to the first type value may also include:

[0170] If the characteristic standard deviation corresponding to the matrix element is greater than a preset threshold value, and the standard deviation characterization value is a first type characterization value, then it is determined that the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition;

[0171] or,

[0172] If the characteristic standard deviation corresponding to the matrix element is less than the preset definition threshold, and the standard deviation characterization value is a second type characterization value, it is determined that the characteristic standard deviation and the standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition.

[0173] In actual use, when enhancing the feature reconstruction value, at least one filter can be used, and the enhancement of each filter is executed in sequence according to the order of the filters (the execution order of the filters can be pre-set by the administrator of the encoding device or decoding device).

[0174] Among them, different indexes (idx) can be set for different filters to distinguish them. When determining the feature mask, different filters can set different feature masks for the same matrix elements. For example, for the filter with index idx, the corresponding feature mask can be:

[0175]

[0176] Among them, mask[idx,c,i,j] is the feature mask set by the filter with index idx for the matrix element with matrix coordinates (c,i,j), Threshold[idx] is the preset threshold corresponding to the filter with index idx, GreaterFlag[idx] is the standard deviation representation value set by the filter with index idx for the matrix element with matrix coordinates (c,i,j), and σ[c,i,j] is the feature standard deviation corresponding to the matrix element with matrix coordinates (c,i,j).

[0177] In the specific implementation, if mask[idx,c,i,j] is true (true and false can also be replaced by 1 and 0, then mask[idx,c,i,j]=1), it means that the filter with index idx will perform feature enhancement on the matrix elements with matrix coordinates (c,i,j). At this time, if there are multiple sets of filters, multiple feature enhancements may be performed on the matrix elements with the same matrix coordinates.

[0178] Step S203: Based on the feature mask, feature enhancement is performed on the feature reconstruction value to obtain an enhanced feature value.

[0179] It should be noted that, based on the feature mask, feature enhancement is performed on the feature reconstruction value to obtain the enhanced feature value, which can be obtained by feature enhancing the feature reconstruction value corresponding to some matrix elements that need to be enhanced in the current image feature according to the feature mask, thereby obtaining the enhanced feature value.

[0180] This embodiment obtains the characteristic standard deviation and standard deviation characterization value corresponding to each matrix element in the current image feature; determines the characteristic mask corresponding to each matrix element in the current image feature according to the characteristic standard deviation, standard deviation characterization value and preset threshold value; based on the characteristic mask, performs feature enhancement on the characteristic reconstruction value to obtain the enhanced characteristic value. Since the corresponding characteristic mask is set for each matrix element in advance according to the characteristic standard deviation and standard deviation characterization value corresponding to each matrix element and the preset threshold value, the part of the matrix elements that need to be enhanced in the feature matrix corresponding to the current image feature are marked by setting the characteristic mask, so that the part of the matrix elements that need to be enhanced can be quickly determined when performing feature enhancement, thereby improving the processing efficiency.

[0181] refer to Figure 7 , Figure 7 FIG. 4 is a flow chart of a third embodiment of an image decoding method according to the present invention.

[0182] Based on the above second embodiment, step S203 of the image decoding method of this embodiment includes:

[0183] Step S2031: taking the matrix element whose corresponding feature mask is a first type value as the target matrix element.

[0184] It should be noted that if the feature mask corresponding to the matrix element is a first type value, it means that the feature reconstruction value corresponding to the matrix element needs to be enhanced. Therefore, the matrix elements in the three-dimensional matrix corresponding to the current image feature can be screened according to the feature mask, and the matrix elements whose corresponding feature mask is the first type value can be used as the target matrix elements.

[0185] Step S2032: enhancing the eigenvalues ​​corresponding to the target matrix elements to obtain enhanced eigenvalues.

[0186] It should be noted that when enhancing the eigenvalues ​​corresponding to the target matrix elements, different methods may be used to enhance the eigenvalues ​​to obtain enhanced eigenvalues.

[0187] In a possible implementation manner of the present application, when enhancing the characteristic reconstruction value corresponding to the target matrix element, the characteristic reconstruction value, the residual reconstruction value and the predicted characteristic value corresponding to the target matrix element may be enhanced in combination with a preset scaling factor. In this case, step S2032 of this embodiment may include:

[0188] Obtaining characteristic reconstruction values, residual reconstruction values ​​and predicted characteristic values ​​corresponding to the target matrix elements;

[0189] Determining a first enhancement value according to a first scaling factor and the residual reconstruction value, and determining a second enhancement value according to a second scaling factor and the predicted characteristic value;

[0190] An enhanced feature value is determined according to the feature reconstruction value, the first enhancement value, and the second enhancement value.

[0191] It should be noted that the first scaling factor and the second scaling factor may be preset scaling factors, wherein different filters may correspond to different first scaling factors and second scaling factors.

[0192] In actual use, the enhanced feature value may be determined based on the first feature enhancement formula according to the feature reconstruction value, the first enhancement value and the second enhancement value.

[0193] The first feature enhancement formula is:

[0194] y_hat_en[c,i,j]=y_hat[c,i,j]+mean_hat[c,i,j]*Scale2[idx]+residual_hat[c,i,j]*Scale1[idx]

[0195] Where (c,i,j) is the matrix coordinate of the target matrix element, idx is the index of the filter, y_hat_en[c,i,j] is the enhanced eigenvalue corresponding to the target matrix element, y_hat[c,i,j] is the eigenreconstructed value corresponding to the target matrix element, residual_hat[c,i,j]*Scale1[idx] is the first enhanced value, mean_hat[c,i,j]*Scale2[idx] is the second enhanced value, mean_hat[c,i,j] is the predicted eigenvalue corresponding to the target matrix element, residual_hat[c,i,j] is the residual reconstructed value corresponding to the target matrix element, Scale1[idx] is the first scaling factor, and Scale2[idx] is the second scaling factor.

[0196] In a possible implementation manner of the present application, for the same filter, when the channels corresponding to the target matrix elements are different, different scaling factors may also be set. In this case, before the step of determining the first enhancement value according to the first scaling factor and the predicted characteristic value, and determining the second enhancement value according to the second scaling factor and the residual reconstruction value in this embodiment, the following may also be included:

[0197] Obtaining the feature channel corresponding to the target matrix element;

[0198] A first scaling factor and a second scaling factor are determined according to the characteristic channel.

[0199] It should be noted that obtaining the characteristic channel corresponding to the target matrix element may be obtaining the matrix coordinates (c, i, j) of the target matrix element, extracting c therein, and determining the characteristic channel corresponding to the target matrix element according to the value of c.

[0200] In actual use, determining the first scaling factor and the second scaling factor according to the characteristic channel may be to obtain a factor channel mapping table corresponding to the currently used filter, and search the factor channel mapping table for the corresponding first scaling factor and second scaling factor according to the characteristic channel. The factor channel mapping table includes a mapping relationship between characteristic channels and scaling factors, and different characteristic channels in the factor channel mapping table may correspond to different first scaling factors and second scaling factors. The factor channel mapping table may be pre-set by a manager of an encoding device or a decoding device.

[0201] At this time, the enhanced feature value can be determined according to the feature reconstruction value, the first enhancement value and the second enhancement value according to the second feature enhancement formula. The second feature enhancement formula can be:

[0202] y_hat_en[c,i,j]=y_hat[c,i,j]+mean_hat[c,i,j]*Scale2[c,idx]+residual_hat[c,i,j]*Scale1[c,idx]

[0203] Where (c,i,j) is the matrix coordinate of the target matrix element, idx is the index of the filter, y_hat_en[c,i,j] is the enhanced eigenvalue corresponding to the target matrix element, y_hat[c,i,j] is the feature reconstruction value corresponding to the target matrix element, residual_hat[c,i,j]*Scale1[c,idx] is the first enhanced value, mean_hat[c,i,j]*Scale2[c,idx] is the second enhanced value, mean_hat[c,i,j] is the predicted eigenvalue corresponding to the target matrix element, residual_hat[c,i,j] is the residual reconstruction value corresponding to the target matrix element, Scale1[c,idx] is the first scaling factor corresponding to the feature channel of the target matrix element, and Scale2[c,idx] is the second scaling factor corresponding to the feature channel of the target matrix element.

[0204] Among them, the administrator of the encoding device or decoding device can also pre-set the corresponding control switch for each channel, and turn off the enhancement of the characteristics of a certain channel by turning off the corresponding control switch. Of course, the feature enhancement of a certain channel can also be turned off by modifying the scaling factor corresponding to the channel in the factor channel mapping table to 0.

[0205] In a possible implementation manner of the present application, when enhancing the feature reconstruction value corresponding to the target matrix element, the preset scaling factor and the feature reconstruction value and the predicted feature value corresponding to the target matrix element may be combined for enhancement. In this case, step S2032 of this embodiment may include:

[0206] Obtaining the eigenreconstructed values ​​and predicted eigenvalues ​​corresponding to the target matrix elements;

[0207] Determining a first enhancement value according to a first scaling factor and the feature reconstruction value, and determining a second enhancement value according to a second scaling factor and the predicted feature value;

[0208] An enhancement feature value is determined according to the first enhancement value and the second enhancement value.

[0209] It should be noted that the first scaling factor and the second scaling factor may be pre-set scaling factors, wherein different filters may correspond to different first scaling factors and second scaling factors. Similarly, for the same filter, different first scaling factors and second scaling factors may also be set according to different characteristic channels of the target matrix element.

[0210] In actual use, the enhanced feature value may be determined according to the first enhanced value and the second enhanced value based on a third feature enhancement formula, wherein the third feature enhancement formula is:

[0211] y_hat_en[c,i,j]=y_hat[c,i,j]*Scale1[idx]+mean_hat[c,i,j]*Scale2[idx]

[0212] Where (c,i,j) is the matrix coordinate of the target matrix element, idx is the index of the filter, y_hat_en[c,i,j] is the enhanced eigenvalue, y_hat[c,i,j]*Scale1[idx] is the first enhanced value, mean_hat[c,i,j]*Scale2[idx] is the second enhanced value, y_hat[c,i,j] is the feature reconstruction value corresponding to the target matrix element, mean_hat[c,i,j] is the predicted eigenvalue corresponding to the target matrix element, Scale1[idx] is the first scaling factor, and Scale2[idx] is the second scaling factor.

[0213] In a possible implementation manner of the present application, when enhancing the characteristic reconstruction value corresponding to the target matrix element, the enhancement may be performed in combination with the preset scaling factor and the characteristic reconstruction value and the residual reconstruction value corresponding to the target matrix element. In this case, step S2032 of this embodiment may include:

[0214] Obtaining characteristic reconstruction values ​​and residual reconstruction values ​​corresponding to the target matrix elements;

[0215] Determining a first enhancement value according to a first scaling factor and the feature reconstruction value, and determining a second enhancement value according to a second scaling factor and the residual reconstruction value;

[0216] An enhancement feature value is determined according to the first enhancement value and the second enhancement value.

[0217] It should be noted that the first scaling factor and the second scaling factor may be pre-set scaling factors, wherein different filters may correspond to different first scaling factors and second scaling factors. Similarly, for the same filter, different first scaling factors and second scaling factors may also be set according to different characteristic channels of the target matrix element.

[0218] In actual use, the enhanced feature value may be determined according to the first enhanced value and the second enhanced value based on the fourth feature enhancement formula, and the fourth feature enhancement formula is:

[0219] y_hat_en[c,i,j]=y_hat[c,i,j]*Scale1[idx]+residual_hat[c,i,j]*Scale2[idx]

[0220] Where (c,i,j) is the matrix coordinate of the target matrix element, idx is the index of the filter, y_hat_en[c,i,j] is the enhanced eigenvalue, y_hat[c,i,j]*Scale1[idx] is the first enhanced value, residual_hat[c,i,j]*Scale2[idx] is the second enhanced value, y_hat[c,i,j] is the feature reconstruction value corresponding to the target matrix element, residual_hat[c,i,j] is the residual reconstruction value corresponding to the target matrix element, Scale1[idx] is the first scaling factor, and Scale2[idx] is the second scaling factor.

[0221] In a possible implementation manner of the present application, when enhancing the feature reconstruction value corresponding to the target matrix element, the enhancement may be performed in combination with a preset scaling factor and the feature reconstruction value and feature standard deviation corresponding to the target matrix element. In this case, step S2032 of this embodiment may include:

[0222] Obtaining the characteristic reconstruction value and characteristic standard deviation corresponding to the target matrix element;

[0223] Determining a first enhancement value according to a first scaling factor and the feature reconstruction value, and determining a second enhancement value according to a second scaling factor and the feature standard deviation;

[0224] An enhancement feature value is determined according to the first enhancement value and the second enhancement value.

[0225] It should be noted that the first scaling factor and the second scaling factor may be pre-set scaling factors, wherein different filters may correspond to different first scaling factors and second scaling factors. Similarly, for the same filter, different first scaling factors and second scaling factors may also be set according to different characteristic channels of the target matrix element.

[0226] In actual use, the enhanced feature value may be determined according to the first enhanced value and the second enhanced value based on the fifth feature enhancement formula, and the fifth feature enhancement formula is:

[0227] y_hat_en[c,i,j]=y_hat[c,i,j]*Scale1[idx]+σ[c,i,j]*Scale2[idx]

[0228] Where (c,i,j) is the matrix coordinate of the target matrix element, idx is the index of the filter, y_hat_en[c,i,j] is the enhanced eigenvalue, y_hat[c,i,j]*Scale1[idx] is the first enhanced value, σ[c,i,j]*Scale2[idx] is the second enhanced value, y_hat[c,i,j] is the feature reconstruction value corresponding to the target matrix element, σ[c,i,j] is the feature standard deviation corresponding to the target matrix element, Scale1[idx] is the first scaling factor, and Scale2[idx] is the second scaling factor.

[0229] In a possible implementation manner of the present application, the feature reconstruction value includes reconstruction values ​​corresponding to different components, such as: brightness reconstruction value and chromaticity reconstruction value. Correspondingly, the enhanced feature value may also include enhancement values ​​of different components, such as: brightness enhancement feature value and chromaticity enhancement feature value. When the feature reconstruction value corresponding to the target matrix element is enhanced, the feature enhancement process of different components may be relatively independent and do not affect each other. At this time, step S2032 of this embodiment may include:

[0230] Enhance the chroma reconstruction value corresponding to the target matrix element to obtain a chroma enhancement eigenvalue;

[0231] The brightness reconstruction value corresponding to the target matrix element is enhanced to obtain a brightness enhancement eigenvalue.

[0232] It should be noted that, when performing feature enhancement, different parameters may be set for the enhancement of different components in the same filter, such as setting different scaling factors.

[0233] In a possible implementation manner of the present application, the enhanced eigenvalue of a certain component may be secondary enhanced by using the eigenreconstruction value of other components. For example, the chroma enhancement eigenvalue is secondary enhanced by using the luminance reconstruction value. Then, after the step of enhancing the chroma reconstruction value corresponding to the target matrix element and obtaining the chroma enhancement eigenvalue described in this embodiment, the following may also be included:

[0234] Determine a first enhancement value according to the brightness reconstruction value and a first scaling factor;

[0235] The chroma enhancement feature value is secondary enhanced according to the first enhancement value.

[0236] It should be noted that the first scaling factor may be a preset scaling factor, wherein different filters may correspond to different first scaling factors.

[0237] In actual use, the chroma enhancement feature value may be secondary enhanced according to the first enhancement value based on the sixth feature enhancement formula, and the sixth feature enhancement formula may be:

[0238] y_hat_chroma_en2[c,i,j]=y_hat_chroma_en[c,i,j]+Scale1[idx]*y_hat_luma[c,i,j]

[0239] Wherein, y_hat_chroma_en2[c,i,j] is the chroma enhancement eigenvalue after secondary enhancement, y_hat_chroma_en[c,i,j] is the chroma enhancement eigenvalue, Scale1[idx]*y_hat_luma[c,i,j] is the first enhancement value, Scale1[idx] is the first scaling factor, and y_hat_luma[c,i,j] is the brightness reconstruction value corresponding to the target matrix element.

[0240] Of course, in a specific implementation, the brightness enhancement eigenvalue may also be enhanced twice by the chroma reconstruction value corresponding to the target matrix element.

[0241] In a possible implementation manner of the present application, in order to enable the decoding device to clearly know whether secondary enhancement is required, the step of determining the first enhancement value according to the brightness reconstruction value and the first scaling factor in this embodiment may include:

[0242] Extract component indication parameters from the image code stream;

[0243] If the component indication parameter is a chroma enhancement type parameter, a first enhancement value is determined according to the luma reconstruction value and a first scaling factor.

[0244] It should be noted that the component indication parameter can be an indication parameter for indicating whether to perform secondary enhancement, and the component enhancement characteristic value that needs to be secondary enhanced. For example, the value of the component indication parameter can be 0-3. If the component indication parameter is 0, it means that no secondary enhancement is required; if the component indication parameter is 1, it means that the chroma enhancement characteristic value needs to be secondary enhanced; if the component indication parameter is 2, it means that the brightness enhancement characteristic value needs to be secondary enhanced; if the component indication parameter is 3, it means that both the brightness enhancement characteristic value and the chroma enhancement characteristic value need to be secondary enhanced.

[0245] It can be understood that if the component indication parameter is a chroma enhancement type parameter, it means that secondary enhancement is required, and the component that needs to be secondary enhanced is a chroma enhancement characteristic value. At this time, the first enhancement value can be determined according to the brightness reconstruction value and the first scaling factor, and then the chroma enhancement characteristic value is secondary enhanced according to the first enhancement value.

[0246] In a possible implementation of the present application, the filter may also use the same scaling factor to perform feature enhancement on all matrix elements in the current image feature. In this case, the feature reconstruction value corresponding to the target matrix element may be enhanced according to the seventh feature enhancement formula, and the seventh feature enhancement formula is:

[0247] y_hat_en[c,i,j]=y_hat[c,i,j]*Scale1[idx]

[0248] Where (c,i,j) is the matrix coordinate of the target matrix element, idx is the index of the filter, y_hat[c,i,j] is the feature reconstruction value corresponding to the target matrix element, y_hat_en[c,i,j] is the enhanced feature value, and Scale1[idx] is the scaling factor corresponding to the filter with index idx.

[0249] In a possible implementation of the present application, different scaling factors may be set to perform feature enhancement on matrix elements of different channels in the current image feature (of course, the scaling factors of some of the same channels are allowed to be the same). In this case, the feature reconstruction value corresponding to the target matrix element may be enhanced according to the eighth feature enhancement formula, and the eighth feature enhancement formula is:

[0250] y_hat_en[c,i,j]=y_hat[c,i,j]*Scale1[c]

[0251] Where (c,i,j) is the matrix coordinate of the target matrix element, idx is the index of the filter, y_hat[c,i,j] is the feature reconstruction value corresponding to the target matrix element, y_hat_en[c,i,j] is the enhanced feature value, and Scale1[c] is the scaling factor corresponding to the c channel.

[0252] In this embodiment, the matrix element whose corresponding feature mask is a first type value is used as the target matrix element; the feature reconstruction value corresponding to the target matrix element is enhanced to obtain the enhanced feature value. Since some matrix elements that need to be feature enhanced are first marked as target matrix elements according to the feature mask, and then the feature reconstruction value of the target matrix element is enhanced, the number of matrix elements that need to be processed in the processing process is reduced, and the execution efficiency of the image decoding method is improved.

[0253] refer to Figure 8 , Figure 8 FIG. 4 is a flow chart of a fourth embodiment of an image decoding method according to the present invention.

[0254] Based on the above first embodiment, step S20 of the image decoding method of this embodiment includes:

[0255] Step S201 ′: extracting syntax application interval parameters from the image code stream, and obtaining feature position information corresponding to the current image feature.

[0256] It should be noted that if the same enhancement method is used to enhance the image features of the entire image block (that is, the grammatical parameters used in the enhancement process are exactly the same), the parameters used in the enhancement process (such as the first scaling factor and the second scaling factor) may be optimal only for some image features in the image block, but not optimal for other subsequent image features, and may even bring negative effects. In order to avoid this phenomenon, the encoding device can set multiple sets of grammatical parameters when encoding, and use different grammatical parameters to enhance the image features at different positions in the image block.

[0257] In actual use, the grammar application interval parameter can be a parameter used to indicate the application range corresponding to each set of grammar parameters, and the feature position information corresponding to the current image feature can include the position of the current image feature in the image block, such as the row and column number.

[0258] Step S202': Determine enhanced grammar parameters according to the feature position information and the grammar application interval parameters.

[0259] It is necessary to explain the pros and cons. Determining the enhanced grammatical parameters based on the feature position information and the grammatical application interval parameters can be to determine the application range of the grammar corresponding to each grammatical parameter based on the grammatical application interval parameters, compare the feature position information with the application range corresponding to each grammatical parameter, determine the application range of the feature position information, and use the grammatical parameters corresponding to the application range of the feature position information as the enhanced grammatical parameters.

[0260] In actual use, a first grammar parameter and a second grammar parameter may be set, a total of two sets of grammar parameters, and then a grammar application interval parameter may be set to distinguish which image features the first grammar parameter and the second grammar parameter are used for feature enhancement. The grammar application interval parameter limitation may be to limit the application range of the grammar parameter in dimensions such as channel range, row, column, oblique column, etc.

[0261] For example: assuming that there are two sets of grammatical parameters, namely the first grammatical parameters and the second grammatical parameters, and the grammatical application interval parameter applyLineNum extracted from the bitstream is K, then for the image features with feature position information in columns 1 to K (or rows or oblique columns), the first grammatical parameters are used for feature enhancement, and the second grammatical parameters are used for feature enhancement from K to the last column.

[0262] Of course, in actual use, more than two sets of grammar parameters may be set. In this case, a grammar application interval parameter may be set for each set of grammar parameters, and then the application range of each set of grammar parameters is determined according to the grammar application interval parameter.

[0263] For example: assuming that there are N sets of grammar parameters in total, there are also N grammar application interval parameters, which can be expressed as applyLineNum(i) (i=1~N), applyLineNum(i) is the grammar application interval parameter corresponding to the i-th set of grammar parameters. At this time, it can be determined that the application range of the first set of grammar parameters is 1 to applyLineNum(1) columns (or rows or oblique columns) of image features, the application range of the second set of grammar parameters is applyLineNum(1)+1 to applyLineNum(1)+applyLineNum(2) columns (or rows or oblique columns) of image features, the application range of the third set of grammar parameters is applyLineNum(2)+1 to applyLineNum(2)+applyLineNum(3) columns (or rows or oblique columns) of image features, and so on.

[0264] The image features of the first row or column are easier to understand, but the oblique columns are more complicated. Fig. 9 This is for illustration but not for limitation. Fig. 9 This is a schematic diagram of the feature reconstruction sequence of this application. Fig. 9 As shown, the reconstruction order used when reconstructing features is from the upper left to the lower right, in an oblique manner. Fig. 9 In the figure, Row indicates the row where the image feature is located, column indicates the column where the image feature is located, the dotted circle is the reconstructed feature (Samples that are already processed), the black circle is the image feature currently being reconstructed (Current sample), and T is the feature offset for each reconstruction (Wave). Fig. 9 It means that the image feature of the 8th column is being reconstructed. Then, based on the feature position information of the current image feature (i.e. the row and column), it can be determined which column the current image feature is located in. Then, based on the column it is located in, it can be determined which set of grammatical parameters to use for feature enhancement.

[0265] Step S203': performing feature enhancement on the feature reconstruction value according to the enhanced syntax parameter to obtain an enhanced feature value.

[0266] It can be understood that the feature reconstruction value is enhanced according to the enhanced syntax parameter to obtain the enhanced feature value by using the enhanced syntax parameter to perform feature enhancement in a feature enhancement manner provided by any embodiment of the above-mentioned image encoding method, which will not be described in detail here.

[0267] This embodiment extracts the syntax application interval parameter from the image code stream and obtains the feature position information corresponding to the current image feature; determines the enhancement syntax parameter according to the feature position information and the syntax application interval parameter; and performs feature enhancement on the feature reconstruction value according to the enhancement syntax parameter to obtain the enhanced feature value. Since the enhancement syntax parameter used for the specific enhancement is determined according to the feature position information of the current image feature and the syntax application interval parameter extracted from the image code stream when performing feature enhancement, different syntax parameters can be applied when performing feature enhancement on image features at different positions, thereby ensuring the effect of feature enhancement as much as possible.

[0268] refer to Fig.10 , Fig.10 FIG. 4 is a flow chart of a first embodiment of an image encoding method according to the present invention.

[0269] In this embodiment, the image encoding method includes the following steps:

[0270] Step S910: extracting features from the image block to be encoded, and using the extracted features as current image features.

[0271] It should be noted that the executor of this embodiment can be the encoding device, and the encoding device can be an electronic device such as a personal computer or a server, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image encoding method of this application is explained by taking the encoding device as an example.

[0272] It should be noted that the image block to be encoded may be an image block obtained by dividing the image data to be encoded, wherein the image data may be divided into only one image block or may be divided into multiple image blocks.

[0273] In actual use, feature extraction is performed on the image block to be encoded, and the extracted features are used as the current image features. The feature extraction is performed on the image block to be encoded using an analysis transformation network, and then the current extracted image features are used as the current image features.

[0274] Step S920: Predicting based on the feature reconstruction value corresponding to the reconstructed feature to obtain a predicted feature value.

[0275] In actual use, prediction is performed based on the feature reconstruction value corresponding to the reconstructed feature, and the predicted feature value can be obtained by using the mean prediction network to perform feature prediction based on the feature reconstruction value corresponding to the reconstructed feature, thereby obtaining the predicted feature value. When the mean prediction network is used for prediction, auxiliary information can also be calculated through a hyperparameter encoding network and input into the mean prediction network, so that the mean prediction network combines the feature reconstruction value corresponding to the reconstructed feature and the auxiliary information for prediction.

[0276] Step S930: Determine the encoding residual coefficient corresponding to the current image feature according to the predicted feature value.

[0277] In actual use, the predicted feature value may be subtracted from the feature value of the current image feature to obtain the original residual value, and then the original residual value may be subjected to residual processing and quantization processing to obtain the encoded residual coefficient.

[0278] Step S940: writing the coding residual coefficient into the image code stream corresponding to the image block to be coded.

[0279] It should be noted that the coding residual coefficient is written into the image code stream corresponding to the image block to be encoded. Then, when image decoding is required, the coding residual coefficient can be directly read from the image code stream, and then the coding residual coefficient is subjected to inverse quantization and residual recovery processing to obtain the residual reconstruction value. Finally, the predicted characteristic value of the mean prediction network and the residual reconstruction value can be combined to determine the characteristic reconstruction value corresponding to the image block to be encoded, so as to facilitate image reconstruction.

[0280] In actual use, the encoding device can also perform operations such as parameter calculation, parameter setting, syntax setting, flag setting, etc. The specific implementation method can be obtained by referring to the content of any embodiment of the above-mentioned image encoding method.

[0281] This embodiment extracts features from the image block to be encoded, and uses the extracted features as the current image features; predicts based on the feature reconstruction values ​​corresponding to the reconstructed features to obtain predicted feature values; determines the coding residual coefficients corresponding to the current image features based on the predicted feature values; and writes the coding residual coefficients into the image code stream corresponding to the image block to be encoded. Since the coding residual coefficients of the current image features are calculated and written into the image code stream during encoding, the coding efficiency of encoding the image data is reduced.

[0282] refer to Fig.11 , Fig.11 FIG. 4 is a flow chart of a second embodiment of an image encoding method according to the present invention.

[0283] Based on the first embodiment of the image encoding method, this embodiment may further include, after step S940:

[0284] Step S950: performing image decoding on the image code stream corresponding to the image block to be encoded to obtain a reconstructed image block.

[0285] It should be noted that after the encoding device completes encoding the image data, it is also necessary to verify the encoding efficiency to ensure that the encoding efficiency is high. At this time, the encoding device can decode the image code stream corresponding to the image block to be encoded to obtain a reconstructed image block.

[0286] Step S960: determining the image coding efficiency according to the reconstructed image block and the image block to be encoded.

[0287] In actual use, the image coding efficiency may be determined based on the reconstructed image block and the image block to be encoded by comparing the reconstructed image block with the image block to be encoded, and calculating the corresponding bit rate and PSNR based on the comparison result, thereby determining the image coding efficiency.

[0288] In a possible implementation of the present application, a preset efficiency threshold may be pre-set. After the image coding efficiency is obtained, the image coding efficiency is compared with the preset efficiency threshold. If the image coding efficiency is less than the preset efficiency threshold, it means that the image coding efficiency at this time is low. At this time, the parameters in the network or model used in the encoding process may be adjusted to try to improve the image coding efficiency.

[0289] In actual use, when decoding the image code stream corresponding to the image block to be encoded, the image decoding method provided by any of the above-mentioned image decoding methods of this application can be used, and this embodiment does not limit this. The encoding device can also perform operations such as parameter calculation, parameter setting, syntax setting, and flag bit setting. The specific implementation method can be obtained by referring to the content of any of the above-mentioned image encoding methods.

[0290] In this embodiment, the image code stream corresponding to the image block to be encoded is decoded to obtain a reconstructed image block; the image coding efficiency is determined according to the reconstructed image block and the image block to be encoded. Since the image code stream is obtained after encoding, the image code stream is also decoded, and the reconstructed image block obtained by image decoding is compared with the coded image block to determine the image coding efficiency, the parameters in various networks used in the encoding process can be adjusted according to the image coding efficiency to improve the image coding efficiency.

[0291] In addition, an embodiment of the present invention further proposes a storage medium, on which an image decoding program and / or an image encoding program is stored. When the image decoding program is executed, the image decoding method as described above is implemented; when the image encoding program is executed, the image encoding method as described above is implemented.

[0292] Reference Fig.12 , Fig.12 This is a structural block diagram of the first embodiment of the image decoding device of the present invention.

[0293] like Fig.12 As shown, the image decoding device proposed in the embodiment of the present invention includes:

[0294] The code stream decoding module 10 is used to decode the image code stream and determine the feature reconstruction value corresponding to the current image feature obtained by decoding;

[0295] A feature enhancement module 20 is used to perform feature enhancement on the feature reconstruction value to obtain an enhanced feature value;

[0296] The image reconstruction module 30 is used to perform a synthesis transformation on the enhanced feature values ​​to obtain a reconstructed image block.

[0297] This embodiment decodes the image code stream and determines the feature reconstruction value corresponding to the current image feature obtained by decoding; performs feature enhancement on the feature reconstruction value to obtain an enhanced feature value; and performs a synthetic transformation on the enhanced feature value to obtain a reconstructed image block. Because the feature reconstruction value is firstly feature enhanced before image reconstruction, and then the image is reconstructed according to the enhanced feature enhancement value, the distortion of the image feature in the process of quantization is reduced, thereby improving the image quality of the reconstructed image.

[0298] In a possible implementation manner of the present application, the current image feature is a three-dimensional feature matrix;

[0299] The feature enhancement module 20 is also used to obtain the feature standard deviation and standard deviation representation value corresponding to each matrix element in the current image feature; determine the feature mask corresponding to each matrix element in the current image feature according to the feature standard deviation, the standard deviation representation value and a preset threshold value; based on the feature mask, perform feature enhancement on the feature reconstruction value to obtain an enhanced feature value.

[0300] In a possible implementation of the present application, the feature enhancement module 20 is also used to set the feature mask corresponding to the matrix element to the first type value if the feature standard deviation and standard deviation representation value corresponding to the matrix element meet the preset enhancement conditions; if the feature standard deviation and standard deviation representation value corresponding to the matrix element do not meet the preset enhancement conditions, then set the feature mask corresponding to the matrix element to the second type value.

[0301] In a possible implementation of the present application, the feature enhancement module 20 is also used to determine whether the feature standard deviation and standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition if the feature standard deviation corresponding to the matrix element is greater than a preset threshold value and the standard deviation characterization value is a first type characterization value; or, if the feature standard deviation corresponding to the matrix element is less than a preset threshold value and the standard deviation characterization value is a second type characterization value, then determine whether the feature standard deviation and standard deviation characterization value corresponding to the matrix element meet the preset enhancement condition.

[0302] In a possible implementation of the present application, the feature enhancement module 20 is further used to use the matrix element whose corresponding feature mask is a first type value as the target matrix element; enhance the feature reconstruction value corresponding to the target matrix element to obtain an enhanced feature value.

[0303] In a possible implementation of the present application, the feature enhancement module 20 is also used to obtain the feature reconstruction value, residual reconstruction value and predicted feature value corresponding to the target matrix element; determine the first enhancement value according to the first scaling factor and the residual reconstruction value, and determine the second enhancement value according to the second scaling factor and the predicted feature value; determine the enhanced feature value according to the feature reconstruction value, the first enhancement value and the second enhancement value.

[0304] In a possible implementation of the present application, the feature enhancement module 20 is further used to obtain a feature channel corresponding to the target matrix element; determine a first scaling factor and a second scaling factor according to the feature channel, and different feature channels correspond to different first scaling factors and second scaling factors.

[0305] In a possible implementation of the present application, the feature enhancement module 20 is also used to obtain the feature reconstruction value and the predicted feature value corresponding to the target matrix element; determine the first enhancement value according to the first scaling factor and the feature reconstruction value, and determine the second enhancement value according to the second scaling factor and the predicted feature value; determine the enhanced feature value according to the first enhancement value and the second enhancement value.

[0306] In a possible implementation of the present application, the feature enhancement module 20 is also used to obtain the feature reconstruction value and the residual reconstruction value corresponding to the target matrix element; determine the first enhancement value according to the first scaling factor and the feature reconstruction value, and determine the second enhancement value according to the second scaling factor and the residual reconstruction value; determine the enhanced feature value according to the first enhancement value and the second enhancement value.

[0307] In a possible implementation of the present application, the feature enhancement module 20 is also used to obtain the feature reconstruction value and the feature standard deviation corresponding to the target matrix element; determine the first enhancement value according to the first scaling factor and the feature reconstruction value, and determine the second enhancement value according to the second scaling factor and the feature standard deviation; determine the enhanced feature value according to the first enhancement value and the second enhancement value.

[0308] In a possible implementation manner of the present application, the feature reconstruction value includes a brightness reconstruction value and a chroma reconstruction value, and the enhanced feature value includes a brightness enhancement feature value and a chroma enhancement feature value;

[0309] The feature enhancement module 20 is further used to enhance the chromaticity reconstruction value corresponding to the target matrix element to obtain a chromaticity enhancement feature value; and enhance the brightness reconstruction value corresponding to the target matrix element to obtain a brightness enhancement feature value.

[0310] In a possible implementation manner of the present application, the feature enhancement module 20 is further configured to determine a first enhancement value according to the brightness reconstruction value and a first scaling factor; and perform secondary enhancement on the chroma enhancement feature value according to the first enhancement value.

[0311] In a possible implementation of the present application, the feature enhancement module 20 is further used to extract a component indication parameter from the image code stream; if the component indication parameter is a chroma enhancement type parameter, the first enhancement value is determined according to the brightness reconstruction value and the first scaling factor.

[0312] In a possible implementation of the present application, the code stream decoding module 10 is also used to decode the image code stream and determine the residual reconstruction value corresponding to the current image feature obtained by decoding; predict according to the feature reconstruction value of the reconstructed feature to obtain the predicted feature value; determine the feature reconstruction value corresponding to the current image feature according to the residual reconstruction value and the predicted feature value.

[0313] In a possible implementation of the present application, the bitstream decoding module 10 is further used to predict based on the enhanced feature value of the reconstructed feature to obtain a predicted feature value; and determine the feature reconstruction value corresponding to the current image feature based on the residual reconstruction value and the predicted feature value.

[0314] In a possible implementation manner of the present application, the enhanced feature value of the reconstructed feature and the enhanced feature value corresponding to the current image feature are enhanced in the same manner or in a different manner.

[0315] In a possible implementation of the present application, a syntax flag is used to determine whether the enhanced feature value of the reconstructed feature is the same as or different from the enhanced feature value corresponding to the current image feature, and the syntax flag is read from an image code stream.

[0316] In a possible implementation of the present application, the feature enhancement module 20 is also used to extract syntax application interval parameters from the image code stream and obtain feature position information corresponding to the current image feature; determine enhanced syntax parameters based on the feature position information and the syntax application interval parameters; and perform feature enhancement on the feature reconstruction value based on the enhanced syntax parameters to obtain an enhanced feature value.

[0317] Reference Fig.13 , Fig.13 FIG. 4 is a structural block diagram of the first embodiment of the image encoding device of the present invention.

[0318] like Fig.13 As shown, the image encoding device proposed in the embodiment of the present invention includes:

[0319] A feature extraction module 110 is used to extract features from the image block to be encoded and use the extracted features as current image features;

[0320] A feature prediction module 120, configured to predict a feature reconstruction value corresponding to a reconstructed feature to obtain a predicted feature value;

[0321] A residual calculation module 130, configured to determine a coding residual coefficient corresponding to the current image feature according to the predicted feature value;

[0322] The parameter writing module 140 is used to write the coding residual coefficient into the image code stream corresponding to the image block to be encoded.

[0323] This embodiment extracts features from the image block to be encoded, and uses the extracted features as the current image features; predicts based on the feature reconstruction values ​​corresponding to the reconstructed features to obtain predicted feature values; determines the coding residual coefficients corresponding to the current image features based on the predicted feature values; and writes the coding residual coefficients into the image code stream corresponding to the image block to be encoded. Since the coding residual coefficients of the current image features are calculated and written into the image code stream during encoding, the coding efficiency of encoding the image data is reduced.

[0324] In a possible implementation manner of the present application, the parameter writing module 140 is further used to perform image decoding on the image code stream corresponding to the image block to be encoded to obtain a reconstructed image block; and determine the image coding efficiency according to the reconstructed image block and the image block to be encoded.

[0325] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0326] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0327] In addition, for technical details not fully described in this embodiment, reference may be made to the image decoding or image decoding method provided in any embodiment of the present invention, and will not be repeated here.

[0328] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0329] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0330] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0331] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image decoding method, It is characterized in that The image decoding method comprises the following steps: Decoding the image code stream and determining a feature reconstruction value corresponding to the current image feature obtained by decoding; Performing feature enhancement on the feature reconstruction value to obtain an enhanced feature value; The enhanced feature values ​​are synthetically transformed to obtain a reconstructed image block.

2. The image decoding method according to claim 1, It is characterized in that The current image feature is a three-dimensional feature matrix; The step of performing feature enhancement on the feature reconstruction value to obtain an enhanced feature value includes: Obtaining the characteristic standard deviation and standard deviation representation value corresponding to each matrix element in the current image feature; Determine a feature mask corresponding to each matrix element in the current image feature according to the feature standard deviation, the standard deviation representation value and a preset threshold value; Based on the feature mask, feature enhancement is performed on the feature reconstruction value to obtain an enhanced feature value.

3. An image decoding device, It is characterized in that The image decoding device comprises the following modules: A code stream decoding module, used to decode the image code stream and determine a feature reconstruction value corresponding to the current image feature obtained by decoding; A feature enhancement module, used to perform feature enhancement on the feature reconstruction value to obtain an enhanced feature value; The image reconstruction module is used to perform a synthetic transformation on the enhanced feature value to obtain a reconstructed image block.

4. A method for encoding an image, It is characterized in that The image encoding method comprises: Extract features from the image block to be encoded, and use the extracted features as current image features; Predicting is performed according to the feature reconstruction value corresponding to the reconstructed feature to obtain the predicted feature value; Determine the encoding residual coefficient corresponding to the current image feature according to the predicted feature value; The coding residual coefficient is written into the image code stream corresponding to the image block to be coded.

5. An image encoding device, It is characterized in that The image encoding device comprises: A feature extraction module is used to extract features from the image block to be encoded and use the extracted features as current image features; A feature prediction module, used to predict according to the feature reconstruction value corresponding to the reconstructed feature to obtain a predicted feature value; A residual calculation module, used to determine the encoding residual coefficient corresponding to the current image feature according to the predicted feature value; The parameter writing module is used to write the coding residual coefficient into the image code stream corresponding to the image block to be encoded.

6. A decoding device, It is characterized in that The decoding device comprises: a processor, a memory, and a decoding program stored in the memory and executable on the processor, wherein the decoding program implements the image decoding method according to any one of claims 1 to 2 when executed by the processor.

7. A coding device, It is characterized in that The encoding device includes: a processor, a memory, and a decoding program and / or encoding program stored in the memory and executable on the processor. When the decoding program is executed by the processor, the image decoding method according to any one of claims 1 to 2 is implemented. When the encoding program is executed by the processor, the image encoding method according to claim 4 is implemented.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores an image decoding program and / or an image encoding program. When the image decoding program is executed, the image decoding method according to any one of claims 1 to 2 is implemented. When the image encoding program is executed, the image encoding method according to claim 4 is implemented.

Images