Image encoding, decoding, and compression methods for both machine and human vision

Abstract

The present invention provides image encoding, decoding, and compression methods for both machine and human vision. The main process comprises: obtaining a first feature map and a second feature map of an image to be encoded; obtaining hyperprior information of the first feature map; obtaining quantization step sizes of said image under different machine vision tasks, and performing quantization to obtain corresponding feature maps; modeling the feature maps as Gaussian distributions, and predicting corresponding means and variances by means of an entropy model; performing encoding layer by layer on the basis of the distribution and probability of each feature map to obtain a feature map binary code stream; and combining a hyperprior information binary code stream and the feature map binary code stream to obtain a compressed image binary code stream. In the present invention, the quantization step size of feature map quantization is adaptively adjusted to form a transmission feature code stream for human vision and different machine vision tasks, allowing information in the feature maps to be used more efficiently, thereby reducing the transmission code rate, ensuring the performance of the machine vision tasks, and balancing the rate-distortion performance for human vision.

Description

Image encoding, decoding, and compression methods for machine and human vision Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an image encoding, decoding, and compression method for both machine and human vision. Background Technology

[0002] In recent years, end-to-end image compression coding technology based on deep learning has developed rapidly. However, many image encoding and transmission scenarios are not designed for human vision, but rather involve machines performing computer vision tasks on decoded images. Therefore, the main goal of end-to-end image compression models is not only to maintain visual quality, but also to maintain the task accuracy of machine tasks at a given bit rate. In 2019, L. Duan published "Video coding for machines: A paradigm of collaborative compression and intelligent analytics" in the journal *IEEE Transactions on Image Processing*, proposing a deep learning-based end-to-end image compression coding scheme for machine vision. Y. Hu's "Towards coding for human and machine vision: A scalable image coding approach," published at the 2020 *IEEE International Conference on Multimedia and Expo*, proposed an image compression coding framework that simultaneously caters to both human and machine vision through scalable coding, providing a wider range of application scenarios for image compression coding. In 2022, H. Choi published "Scalable Image Coding for Humans and Machines" in the journal IEEE Transactions on Image Processing. He proposed an image coding scheme based on fixed feature channel grouping for machine vision and human vision. The scheme selects the number of transmitted feature groups according to the complexity of the machine task, which greatly improves the task accuracy of machine tasks at a given bit rate with minimal impact on human vision performance.

[0003] However, since the feature channel grouping is fixed, the specified feature channel group is not the optimal feature channel group for different computer vision tasks. This results in redundancy in the transmitted bitstream and the performance of the machine task is not optimal. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the purpose of this invention is to provide an image encoding, decoding and compression method for both machine and human vision.

[0005] A first aspect of the present invention provides an image coding method for machine and human vision, comprising:

[0006] For the image to be compressed, a multi-channel feature map of the image is generated using an encoding convolutional neural network, which serves as the first feature map; the first feature map is quantized with a quantization step size of 1 to obtain a feature map oriented towards human visual encoding, which serves as the second feature map;

[0007] For the first feature map, the super-prior information is extracted by the super-prior coding convolutional neural network and the super-prior information binary code stream is obtained. Then, the super-prior information binary code stream is decoded by the super-prior decoding convolutional neural network to generate reconstructed super-prior information.

[0008] The quantization step size for machine vision coding is estimated using the reconstructed prior information and adjusted to an odd number. The first feature map is then quantized into quantized feature maps for N machine vision tasks based on the quantization step size, where N is a positive integer.

[0009] Using the reconstructed prior information, the distribution of the quantized feature map and the distribution of the second feature map for the Nth to the 1st machine vision task are predicted sequentially using an entropy model; probability estimation is performed based on the predicted distribution, and arithmetic encoding is performed based on the estimated probability to obtain a layer-by-layer feature map binary code stream, which is then merged to obtain the feature map binary code stream.

[0010] The binary code stream of the prior information and the binary code stream of the feature map are merged to obtain the compressed image binary code stream.

[0011] Preferably, for the first feature map, extracting prior information using a priori coding convolutional neural network and obtaining a priori information binary code stream, and then decoding the priori information binary code stream using a priori decoding convolutional neural network to generate reconstructed prior information, includes:

[0012] The super-prior feature map is obtained from the first feature map by a super-prior coding convolutional neural network;

[0013] The super-prior feature map is quantized to obtain a quantized super-prior feature map;

[0014] The quantized hyperprior feature map is subjected to distribution estimation of mutually independent features, and the distribution estimation is used to perform arithmetic encoding on the quantized hyperprior feature map to obtain a hyperprior information binary code stream.

[0015] Arithmetic decoding is used to recover the prior information from the prior information binary code stream;

[0016] Reconstructed hyperprior information is obtained from the recovered hyperprior information by using a hyperprior decoding convolutional neural network;

[0017] The super-prior decoding convolutional neural network is a deconvolutional neural network corresponding to the super-prior encoding convolutional neural network.

[0018] Preferably, the step of estimating the quantization step size for machine vision coding using the reconstructed prior information and adjusting it to an odd number, and quantizing the first feature map into quantized feature maps for N machine vision tasks according to the quantization step size, where N is a positive integer, including:

[0019] Based on the information requirements of machine vision tasks, the indices i = 1, ..., N for N machine vision tasks are defined. The smaller the index, the greater the information required by the machine vision task. The machine vision tasks include image classification, object detection, semantic segmentation, and instance segmentation.

[0020] The quantization step size relationship between N machine vision tasks is obtained by using the reconstructed prior information;

[0021] Based on the correspondence between quantization step sizes of different machine vision tasks, the quantization step sizes for machine vision-oriented coding of different tasks are obtained.

[0022] The first feature map is quantized using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks.

[0023] Preferably, obtaining the quantization step size relationship between N machine vision tasks using the reconstructed prior information includes:

[0024] For i = 1, ..., N, a quantized convolutional neural network is used to estimate the quantization step size relationship q of the first feature map between the i-th machine vision task and the (i+1)-th machine vision task. i Specifically:

[0025] The reconstructed prior information Conv1 is a convolutional layer with a kernel size of 3×3. 3×3 A transformation is performed, using LeakyReLU as the nonlinear mapping layer, to obtain the latent quantization step size relationship information.

[0026] The LeakyReLU function is as follows:

[0027] Where α is a positive parameter used to control the gradient of negative values;

[0028] The relationship information of the quantization step size of the subsurface Upsampling is performed by sequentially using the DeConv1 deconvolution layer of the SoftPlus nonlinear mapping layer. 3×3 and the deconvolutional layer DeConv2 using LeakyReLU nonlinear mapping layers 3×3 The rough quantization step size relationship is obtained.

[0029] The SoftPlus function expression is as follows:

[0030] SoftPlus(x) = log(1 + e) x )

[0031] The coarse quantization step size relationship Conv2 convolutional layer with a kernel size of 3×3 as input 3×3 Using a SoftPlus nonlinear mapping layer and adding a bias of 1, the quantization step size relationship is ensured to be no less than 1, thus obtaining a continuous quantization step size relationship.

[0032] Adjust the relationship of the continuous quantization step size By approximating an odd number for the quantization step size, we obtain the quantization step size relationship q between the i-th machine vision task and the (i+1)-th machine vision task. i Specifically:

[0033] in, Return the largest positive integer not greater than q.

[0034] Preferably, obtaining the quantization step size for machine vision encoding of different tasks based on the quantization step size correspondence between different machine vision tasks includes:

[0035] For i = 1, ..., N, estimate the quantization step size Q of the first feature map under the i-th machine vision task. i :

[0036] Preferably, the step of quantizing the first feature map using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks specifically involves:

[0037] For i = 1, ..., N, obtain the quantized feature map corresponding to the i-th machine vision task. for:

[0038] Where y is the first feature map, and Round(y) returns the result after rounding the decimal part of y.

[0039] Preferably, the step of using reconstructed prior information to sequentially predict the distribution of the quantized feature maps and the distribution of the second feature map for the Nth to the 1st machine vision task using an entropy model, performing probability estimation based on the predicted distribution, performing arithmetic encoding based on the estimated probability to obtain a layer-by-layer feature map binary code stream, and merging to obtain the feature map binary code stream includes:

[0040] Quantize the feature map of the Nth machine vision task The parameterized model is a Gaussian distribution, and the entropy model is used to predict the mean and variance of the Gaussian distribution using the reconstructed prior information;

[0041] Based on the predicted mean and variance of the Gaussian distribution, the quantized feature map of the Nth machine vision task is obtained. Probability estimation is performed, and arithmetic encoding is performed based on the estimated probability to obtain the binary code stream of the first layer feature map;

[0042] The entropy model is used sequentially to predict the mean and variance of the Gaussian distribution of the quantized feature maps of the N-1th to the 1st machine vision task using the reconstructed prior information. Probability estimation is performed based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is performed based on the estimated probability to obtain the binary code stream of the feature maps of the 2nd to Nth layers.

[0043] Using the reconstructed prior information, the mean and variance of the Gaussian distribution of the second feature map are predicted by the entropy model. Probability estimation is performed based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is performed based on the estimated probability to obtain the binary code stream of the N+1 layer feature map.

[0044] Following the order of layers 1 to N+1, the binary code streams of the feature maps in layer N+1 are merged to obtain the binary code stream of the feature maps.

[0045] According to a second aspect of the present invention, an image decoding method for machine and human vision is provided, which corresponds to an image encoding method for decoding any one of the machine and human vision methods, comprising:

[0046] The image binary code stream is decomposed and compressed to obtain a priori information binary code stream and a feature map binary code stream; the feature map binary code stream is decomposed to obtain an N+1 layer feature map binary code stream.

[0047] Arithmetic decoding is used to recover the prior information from the binary code stream of the prior information; the deconvolutional neural network of the corresponding prior coding convolutional neural network is used as the prior decoding convolutional neural network to obtain the reconstructed prior information from the recovered prior information.

[0048] A quantized convolutional neural network is employed to estimate the quantization step size corresponding to N machine vision tasks from the reconstructed prior information, and adjust it to an odd number. An entropy model is used to utilize the reconstructed prior information, combined with the quantization step size of each machine vision task layer, to predict the mean and variance of the Gaussian distribution of the quantized feature maps for the Nth to 1st machine vision tasks. Based on the mean and variance of the Gaussian model, the binary code stream of the feature maps is decoded layer by layer from layer 1 to layer N to obtain the reconstructed quantized feature maps for the Nth to 1st machine vision tasks. An entropy model is then used to predict the mean and variance of the Gaussian distribution of the second feature map using the reconstructed prior information. Based on the mean and variance of the Gaussian model, the binary code stream of the feature map at layer N+1 is decoded to obtain the reconstructed second feature map.

[0049] The reconstructed second feature map is then passed through a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain a reconstructed image for human vision.

[0050] The reconstructed feature maps of N machine vision tasks are passed through the corresponding encoding convolutional neural network and the decoding convolutional neural network to obtain the reconstructed image for the machine vision task.

[0051] According to a third aspect of the present invention, an image compression method for machine and human vision is provided, comprising:

[0052] Obtain the image to be compressed;

[0053] The image is compressed using any of the machine- and human-vision-oriented image coding methods described above to obtain a compressed image binary stream;

[0054] The image decoding method for machine and human vision is used to decompress the compressed image binary stream to obtain a reconstructed image for human and machine vision.

[0055] Preferably, the parameters of the encoding convolutional neural network, decoding convolutional neural network, super-prior encoding convolutional neural network, super-prior decoding convolutional neural network, and entropy model are obtained through rate-distortion optimization.

[0056] Among them, distortion x is the image to be compressed. This is a reconstructed image; λ>0 is a preset parameter used to balance distortion and bitrate; the bitrate R is based on the second feature map. The Gaussian distribution of each element is calculated as follows:

[0057] in, It is the characteristic of the k-th element. It is the predicted distribution of the k-th element. To reconstruct the estimated distribution of the prior information, θ is used to estimate the number of codewords required for transmitting the reconstructed prior information. k ={φ k ,ψ k} is a set of learnable parameters.

[0058] Preferably, the parameters of the quantized convolutional neural network for N machine vision tasks are obtained through optimization of bitrate and task accuracy:

[0059] in, Let λ represent the loss function related to the accuracy of the i-th machine vision task. i >0 is a preset parameter used to balance the bitrate and task accuracy for the i-th machine vision task. i Let ω be the bitrate of the i-th machine vision task. i >0 is a preset parameter used to balance different machine vision tasks.

[0060] According to a fourth aspect of the present invention, a terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any of the following image encoding methods for machine and human vision: one, one image decoding method for machine and human vision, or one image compression method for machine and human vision.

[0061] According to a fifth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform any of the following image encoding methods for machine and human vision: one described above, another described above, an image decoding method for machine and human vision, or any of the following image compression methods for machine and human vision.

[0062] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0063] The image encoding, decoding, and compression method for machine and human vision in this embodiment of the invention adaptively adjusts the quantization step size of feature map quantization to form a transmission feature code stream for human vision and different machine vision tasks. This can utilize the information in the feature map more efficiently, reduce the transmission bit rate while ensuring the performance of machine vision tasks, and balance the rate-distortion performance for human vision.

[0064] The image encoding, decoding, and compression methods for machine and human vision in the embodiments of the present invention enable a single bitstream to perform multiple different machine vision tasks and achieve performance similar to that of a single machine vision task.

[0065] The image encoding, decoding, and compression method for machine and human vision in this embodiment of the invention can generate a truncated single binary bitstream by quantizing the step size, forming reconstructed images for different tasks of human and machine vision, adapting to terminals performing different tasks, eliminating the need to switch between different bitstream versions for multiple different tasks, improving user experience, and reducing the overall bandwidth requirements of network transmission.

[0066] The image encoding, decoding, and compression methods for machine and human vision in this embodiment of the invention can more efficiently utilize feature maps to perform downstream image tasks. By controlling the quantization step size of the feature maps, the transmission bit rate can be reduced to the minimum while meeting the task performance indicators. Attached Figure Description

[0067] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0068] Figure 1 is a flowchart of an image coding method for machine and human vision according to an embodiment of the present invention;

[0069] Figure 2 is a flowchart of an image coding method for machine and human vision in a preferred embodiment of the present invention;

[0070] Figure 3 is a flowchart of an image decoding method for machine and human vision in a preferred embodiment of the present invention;

[0071] Figure 4 is a flowchart of an image compression method for machine and human vision in a preferred embodiment of the present invention;

[0072] Figure 5 is a flowchart of a training method for an image compression model oriented towards machine and human vision in a preferred embodiment of the present invention. Detailed Implementation

[0073] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0074] Referring to Figure 1, a flowchart of image encoding for machine and human vision according to an embodiment of the present invention is shown. This embodiment provides an image encoding method for machine vision, including steps S100-S500. Specifically:

[0075] S100. For the image to be compressed, a multi-channel feature map of the image is generated using an encoding convolutional neural network, which serves as the first feature map; the first feature map is quantized with a quantization step size of 1 to obtain a feature map oriented towards human visual encoding, which serves as the second feature map;

[0076] S200. For the first feature map obtained in S100, extract the super-prior information through the super-prior coding convolutional neural network, obtain the super-prior information binary code stream, and decode the super-prior information binary code stream to generate the reconstructed super-prior information.

[0077] S300. Utilize the reconstructed prior information obtained in S200 to estimate the quantization step size for machine vision coding, and adjust it to an odd number to satisfy the embedded quantization condition, eliminate codeword redundancy, and improve codeword utilization. Based on the quantization step size, quantize the first feature map obtained in S100 into quantized feature maps for N machine vision tasks, where N is a positive integer.

[0078] S400. Using the reconstructed prior information obtained in S200, the entropy model is used to predict the distribution of the quantized feature map and the second feature map of the Nth machine vision task to the 1st machine vision task in sequence; probability estimation is performed based on the predicted distribution, and arithmetic encoding is performed based on the estimated probability to obtain the layer-by-layer feature map binary code stream, which is then merged to obtain the feature map binary code stream.

[0079] S500. Merge the binary code stream of the prior information obtained in S200 and the binary code stream of the feature map obtained in S400 to obtain the compressed image binary code stream.

[0080] In step S100 of the above embodiment, the first feature map mentioned is a three-dimensional feature map of the image to be compressed. The dimensions of the three-dimensional feature map include feature channels, feature height, and feature width. In S400, it is mentioned that the entropy model network is implemented using existing technologies such as super-prior networks and spatial context models. For example, by using a super-prior decoding convolutional neural network, the super-prior feature map is recovered from the super-prior information, corresponding to the mean, variance, etc. of a Gaussian distribution. When encoding the feature maps and second feature maps corresponding to each machine vision task, a progressive coding scheme is used to obtain a layered bitstream. The progressive coding scheme is implemented using existing technologies.

[0081] The embodiments of the present invention described above can efficiently extract the features and information most needed by machine vision, while reducing redundant feature transmission for machine vision tasks. This reduces image compression bitrate overhead and effectively improves the accuracy of downstream tasks. Furthermore, by employing progressive coding, the encoded bitstream can be layered, allowing a single bitstream to perform multiple machine vision tasks simultaneously, resulting in higher efficiency and reduced bitstream transmission overhead under multi-tasking conditions.

[0082] Figure 2 is a flowchart of an image encoding method for machine and human vision according to a preferred embodiment of the present invention. In this preferred embodiment, the image encoding for machine vision may include more detailed steps S101-S109, specifically:

[0083] S101, 3D feature extraction: The image to be compressed is processed through a coded convolutional neural network to obtain a feature map, thus obtaining 3D features, i.e., the first feature map;

[0084] S102, Extraction and quantization of prior information: The prior information is obtained by using a prior coding convolutional neural network on the first feature map; the prior information is rounded and quantized into integers;

[0085] S103, Encoding of Super Prior Information: Establish a distribution estimate of the mutual independence between each feature of the super prior information, and use this distribution estimate to perform arithmetic encoding on the super prior information to obtain the binary code stream of the super prior information;

[0086] S104, Decoding of Super-Prior Information: Arithmetic decoding of the binary code stream of super-prior information is performed to obtain the recovered super-prior information features; a super-prior decoding convolutional neural network is used to obtain the super-prior feature map from the recovered super-prior information.

[0087] S105, Quantization: Round the first feature map to an integer to obtain the second feature map;

[0088] S106, Machine vision task quantization: Based on the recovered prior information, the quantization step size relationship between different task layers is obtained through a quantized convolutional neural network; then, based on the obtained quantization step size correspondence of different task layers, the quantization step size under different tasks is obtained; finally, the first feature map is quantized using the quantization step size oriented towards machine vision encoding to obtain feature maps corresponding to different machine vision tasks.

[0089] S107, Probability Entropy Modeling: The distribution of each feature in the feature map is parameterized and modeled as a Gaussian distribution, and the mean and variance of the corresponding Gaussian distribution are predicted using prior information and context model.

[0090] S108, 3D feature progressive coding: Based on the mean and variance of the Gaussian distribution of all predicted feature maps, the features of all feature maps are probabilistically estimated. Using existing progressive coding techniques, all feature maps are encoded layer by layer and merged to obtain the feature binary code stream of the 3D features.

[0091] S109, the feature binary code stream is further merged with the prior feature binary code stream to obtain the compressed image binary code stream.

[0092] To ensure optimal execution results for each step, some embodiments of the present invention provide preferred solutions for S200, S300, and S400, respectively.

[0093] In one embodiment of the present invention, a preferred solution for implementing step S200 is provided, the specific steps of which are as follows:

[0094] S21, Obtain the super-prior feature map from the first feature map through the super-prior coding convolutional neural network;

[0095] S22, quantize the super-prior feature map obtained in S21 to obtain the quantized super-prior feature map;

[0096] S23, perform distribution estimation on the quantized super-prior feature map obtained in S22 to estimate the independent features, and use the distribution estimation to perform arithmetic coding on the quantized super-prior feature map to obtain the super-prior information binary code stream.

[0097] S24 uses arithmetic decoding to recover the prior information from the prior information binary code stream obtained in S23;

[0098] S25, reconstructed prior information is obtained by using the prior information recovered from S24 through the prior decoding convolutional neural network;

[0099] Among them, the super-prior decoding convolutional neural network is the deconvolutional neural network corresponding to the super-prior encoding convolutional neural network.

[0100] In the above embodiments, steps S21-S25 encode the prior information of the first and second feature maps to be encoded, reducing the encoding overhead of the first and second feature maps and improving the overall encoding efficiency.

[0101] In one embodiment of the present invention, a preferred method for implementing step S300 is provided, the specific steps of which are as follows:

[0102] S31. Based on the information required by the machine vision task, the indices i = 1, ..., N of the N machine vision tasks are defined. The smaller the index, the greater the information required by the machine vision task. The machine vision tasks include image classification, object detection, semantic segmentation and instance segmentation.

[0103] S32, using the reconstructed prior information determined by S200, obtain the quantization step size relationship between N machine vision tasks;

[0104] S33, based on the quantization step size correspondence between different machine vision tasks, obtain the quantization step size for machine vision-oriented coding of different tasks;

[0105] S34. The first feature map is quantized using the quantization step size for machine vision coding obtained in S33, resulting in quantized feature maps corresponding to N machine vision tasks.

[0106] In the above embodiments, the multi-feature map quantization implemented in steps S31-S34 can enable a single bitstream to transmit the content required for multiple visual tasks, thereby expanding the application scope of the above method.

[0107] In a preferred embodiment, step S32 is performed to obtain the quantization step size relationship between N machine vision tasks using reconstructed prior information, including:

[0108] For i = 1, ..., N, a quantized convolutional neural network is used to estimate the quantization step size relationship q of the first feature map between the i-th machine vision task and the (i+1)-th machine vision task. i Specifically:

[0109] Reconstructing prior information Conv1 is a convolutional layer with a kernel size of 3×3. 3×3 A transformation is performed, using LeakyReLU as the nonlinear mapping layer, to obtain the latent quantization step size relationship information.

[0110] The LeakyReLU function is as follows:

[0111] Where α is a positive parameter used to control the gradient of negative values;

[0112] Information on the step size relationship of subsurface quantization Upsampling is performed by sequentially using the DeConv1 deconvolution layer of the SoftPlus nonlinear mapping layer. 3×3 and the deconvolutional layer DeConv2 using LeakyReLU nonlinear mapping layers 3×3 The rough quantization step size relationship is obtained.

[0113] The expression for the SoftPlus function is as follows: SoftPlus(x) = log(1 + e^x) x )

[0114] Relationship between coarse quantization step size Conv2 convolutional layer with a kernel size of 3×3 as input 3×3 Using a SoftPlus nonlinear mapping layer and adding a bias of 1, the quantization step size relationship is ensured to be no less than 1, thus obtaining a continuous quantization step size relationship.

[0115] Adjusting the relationship of continuous quantization step size By making the quantization step size approximately odd, the embedded quantization condition is satisfied, codeword redundancy is eliminated, and codeword utilization is improved, thus obtaining the quantization step size relationship q between the i-th machine vision task and the (i+1)-th machine vision task. i The formula is:

[0116] in, Return the largest positive integer not greater than q.

[0117] In a preferred embodiment, step S33 is performed to obtain the quantization step size for machine vision encoding of different tasks based on the quantization step size correspondence between different machine vision tasks, including:

[0118] For i = 1, ..., N, estimate the quantization step size Q of the first feature map under the i-th machine vision task. i :

[0119] In a preferred embodiment, step S34 is performed, whereby the first feature map is quantized using a quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks, specifically:

[0120] For i = 1, ..., N, obtain the quantized feature map corresponding to the i-th machine vision task. for:

[0121] Where y is the first feature map, and Round(y) returns the result after rounding the decimal part of y.

[0122] In the above embodiments of the present invention, by adjusting the feature map quantization step size under machine vision tasks, the present invention can efficiently retain the information that machine vision needs more, reduce redundant information for machine vision, reduce image compression bitrate overhead, and effectively improve the accuracy of downstream tasks.

[0123] In a preferred embodiment of the present invention, S400 is implemented, and the specific process is as follows:

[0124] S41, quantize the feature map of the Nth machine vision task. The parameterized model is a Gaussian distribution, and the entropy model is used to predict the mean and variance of the Gaussian distribution by reconstructing the prior information.

[0125] The quantized feature map of the Nth machine vision task is based on the predicted Gaussian distribution mean and variance. Probability estimation is performed, and arithmetic encoding is performed based on the estimated probabilities to obtain the binary code stream of the first layer feature map;

[0126] S42, the entropy model is used to predict the mean and variance of the Gaussian distribution of the quantized feature maps of the N-1th to the 1st machine vision task using the reconstructed prior information. The probability is estimated based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is performed based on the estimated probability to obtain the binary code stream of the feature maps of the 2nd to Nth layers.

[0127] S43, using the entropy model to reconstruct the prior information, predicts the mean and variance of the Gaussian distribution of the second feature map, performs probability estimation based on the predicted mean and variance of the Gaussian distribution, and performs arithmetic coding based on the estimated probability to obtain the binary code stream of the N+1 layer feature map;

[0128] S44. Merge the binary code streams of the feature maps of layers N+1 in the order of layers 1 to N+1 to obtain the binary code streams of the feature maps.

[0129] The above embodiments of the present invention use the idea of ​​progressive coding, the encoded bitstream can be layered, and a single bitstream can perform multiple machine vision tasks simultaneously, which has higher efficiency and reduces the bitstream transmission overhead under multiple tasks.

[0130] Based on the same inventive concept, another embodiment of the present invention provides an image decoding method for machine and human vision, the flowchart of which is shown in Figure 3, including:

[0131] S201, Bitstream decomposition: Decompose the compressed image binary bitstream to obtain the feature map binary bitstream and the prior information binary bitstream; decompose the feature map binary bitstream to obtain the N+1 layer feature map binary bitstream;

[0132] S202, Decoding of Super Prior Information: Arithmetic decoding is performed on the binary code stream of super prior information to obtain the recovered super prior information; a super prior decoding convolutional neural network is used on the recovered super prior information to obtain the reconstructed super prior information;

[0133] S203, Probabilistic Entropy Modeling: The structure and parameter values ​​of this module are exactly the same as those of the probabilistic entropy-based modeling in the embodiment shown in Figure 2. Using an entropy model, the mean and variance of the Gaussian distribution of the quantized feature maps for the Nth to 1st machine vision tasks are predicted one by one using reconstructed prior information. Based on the mean and variance of the Gaussian model, the binary code stream of the feature maps is decoded layer by layer from layer 1 to layer N to obtain the quantized feature maps corresponding to the Nth to 1st machine vision tasks. Using an entropy model, the mean and variance of the Gaussian distribution of the second feature map are predicted using reconstructed prior information. Based on the mean and variance of the Gaussian model, the binary code stream of the N+1th layer feature map is decoded to obtain the second feature map.

[0134] S204, Machine Vision Task Quantization Step Size Prediction: This module has the same structure as the part in S7 that calculates the quantization step size. It predicts the quantization step size of the feature maps corresponding to each machine vision task from the recovered reconstructed prior information. Specifically, it uses a quantized convolutional neural network to estimate the quantization step size corresponding to N machine vision tasks from the reconstructed prior information and adjusts it to an odd number.

[0135] S205, Progressive Decoding of 3D Features: Based on the mean and variance of the Gaussian model in S203 and the quantization step size of the feature map corresponding to the machine vision task in S204, progressive decoding is performed on the binary code stream of the feature map. Progressive decoding uses existing technology to obtain the recovered feature map corresponding to the machine vision task and the second feature map corresponding to human vision layer by layer.

[0136] S206, Image Reconstruction: The second feature map reconstructed in S205 is passed through the decoding convolutional neural network of the corresponding encoding convolutional neural network to obtain a reconstructed image for human vision.

[0137] The N machine vision task reconstruction feature maps obtained in S205 are used to obtain the reconstructed image for the machine vision task through the corresponding encoding convolutional neural network decoding convolutional neural network.

[0138] The image decoding method for machine vision and human vision based on adaptive quantization step size in this embodiment corresponds to the image encoding method for machine vision shown in Figure 2. Parts not described in detail above can be referred to the corresponding descriptions in the embodiment shown in Figure 2, and will not be repeated here.

[0139] Based on the same inventive concept, other embodiments of the present invention provide an image compression method for machine and human vision, the main steps of which are as follows:

[0140] The first step is to obtain the image to be compressed;

[0141] The second step is to use an image encoding method oriented towards machine and human vision from the above embodiments to compress the image and obtain a compressed image binary code stream.

[0142] The third step involves using the image decoding method for machine and human vision described in the above embodiments to decompress the compressed image binary stream and obtain a reconstructed image for human and machine vision.

[0143] Furthermore, in one specific embodiment, an image compression method oriented towards machine and human vision is used to compress all images in the COCO2017 test set. The flowchart of this method is shown in Figure 4, and includes:

[0144] S301, Using the extraction module: sequentially obtain the images to be compressed;

[0145] S302, Using an encoder: Encode the image to be compressed according to the machine vision-oriented image encoding method in the above embodiment to obtain a compressed image binary code stream.

[0146] Specifically, the number of channels of the extracted 3D features is 192, and the downsampling factor is 16; the number of machine vision tasks set is 2, namely object detection task and semantic segmentation task.

[0147] S303, using a decoder: decompress the compressed image binary stream according to the image decoding method for machine vision, to obtain partially reconstructed images for two types of downstream image tasks and reconstructed images for human vision.

[0148] The compression method in the above embodiments of the present invention is based on the image encoding method for machine and human vision shown in Figures 1 and 2 and the image decoding method for machine and human vision shown in Figure 3. For the parts not described in detail above, please refer to the corresponding descriptions in the above embodiments, and they will not be repeated here.

[0149] The compression method described in the above embodiments of the present invention can efficiently extract the information most needed by machine vision, while reducing redundant information for machine vision, thereby reducing image compression bitrate overhead and effectively improving the accuracy of downstream tasks. Furthermore, the bitstream obtained through progressive coding can be layered, and a single bitstream can simultaneously perform multiple machine vision tasks, resulting in higher efficiency and reduced bitstream transmission overhead under multi-tasking conditions.

[0150] The image compression method described above can be improved by training a model. For example, the training set consists of all images from the COCO2017-train data set, and the trained model can be used for image encoding and decoding in machine vision. Specifically, as shown in Figure 5, in a preferred embodiment, the image compression method includes:

[0151] S401, 3D Feature Extraction: Randomly crop the images in the training set to a resolution of 256×256, and input them into an encoding convolutional neural network to obtain 3D features. The encoding convolutional neural network consists of three residual blocks, one attention mechanism module, three residual blocks, one convolutional layer, and one attention mechanism module cascaded sequentially. The decoding neural network can also be other network structures, such as a four-layer convolutional cascade. The number of 3D feature channels is 128 or 192, and the height and width are both 16. Of course, the image resolution and 3D feature parameters described above in this embodiment can be adjusted according to actual conditions and are not limited to those described in this step. The same applies to the parameters in the following steps.

[0152] S402, Extraction and Noise Addition of Prior Information: The 3D features are input into a priori encoding convolutional neural network to obtain prior features. To avoid gradient problems caused by quantization, noise with a uniform distribution from -0.5 to 0.5 is added to the prior features to simulate actual quantization. The priori encoding convolutional neural network consists of three cascaded convolutional layers. Of course, the priori encoding neural network can also be other network structures, such as three cascaded residual modules, and is not limited to the second convolutional neural network in this step.

[0153] S403, Super-prior code rate estimation: Establish a distribution estimate that is independent of each feature based on the super-prior information, and use this distribution estimate to obtain the super-prior code rate estimate;

[0154] S404, Acquisition of Super Prior Information: Input the noise-added super prior information into the super prior decoding convolutional neural network to obtain the super prior information, wherein: the super prior decoding convolutional neural network is composed of three convolutional layers cascaded together;

[0155] S405, Machine vision quantization step size prediction: Input the noise-added prior information into the quantization convolutional neural network to obtain the feature map quantization step size relationship under each machine vision task, further obtain the feature map quantization step size under each machine vision task, and make the quantization step size of each layer satisfy the odd multiple relationship by quantizing the result of the quantization convolutional neural network to an odd number. Here, to avoid the gradient problem caused by the quantization operation, the gradient is set to 1 during the quantization operation.

[0156] S406, Feature Noise Addition: To avoid gradient problems caused by quantization, noise with a uniform distribution from -0.5Q to 0.5Q is added to the 3D features according to the corresponding quantization step size Q of the feature map to simulate actual quantization; if it is human vision, then Q=1;

[0157] S407, Gaussian distribution probability modeling: The structure of this module is exactly the same as the probability entropy modeling in the embodiment shown in Figure 2; it uses prior information and a spatial context-based autoregressive model to predict the mean and variance of the Gaussian distribution of each group.

[0158] S408, Feature code rate estimation: The feature code rate is estimated based on the mean and variance of the predicted Gaussian distribution.

[0159] S409, Total Bitrate Estimation: Add the prior feature bitrate estimate and the feature bitrate estimate to obtain the total bitrate estimate;

[0160] S410, Image Reconstruction: The noisy 3D features are input into a decoding convolutional neural network to obtain the reconstructed image. The decoding convolutional neural network consists of an attention mechanism module, three residual blocks, another attention mechanism module, three residual blocks, and an upsampling convolutional layer cascaded sequentially. This decoding convolutional neural network can also be other network structures, corresponding to the encoding convolutional neural network. The aforementioned image compression method for machine vision also needs to satisfy the image compression model serving human vision. This can be further optimized in stages using stochastic gradient descent. In the first stage, the machine vision quantization step size prediction module is disabled, and the quantization step size is set to Q=1. Only the encoding convolutional neural network, decoding convolutional neural network, super-prior encoding convolutional neural network, super-prior decoding convolutional neural network, and entropy model are optimized. The loss function for the first stage is:

[0161] Among them, distortion x is the image to be compressed. This is a reconstructed image; λ>0 is a preset parameter used to balance distortion and bitrate; the bitrate R is based on the second feature map. The Gaussian distribution of each element is calculated as follows:

[0162] in, It is the characteristic of the k-th element. It is the predicted distribution of the k-th element, θ k ={φ k ,ψ k} is a set of learnable parameters.

[0163] In the second stage, the encoding convolutional neural network, the decoding convolutional neural network, the super-prior encoding convolutional neural network, the super-prior decoding convolutional neural network, and the entropy model are fixed. The parameters of the machine vision quantization step size prediction module are optimized using the following formula:

[0164] in, Let λ represent the loss function related to the accuracy of the i-th machine vision task. i >0 is a preset parameter used to balance the bitrate and task accuracy for the i-th machine vision task, where R is the bitrate of the i-th machine vision task. iConsistent with the first-stage bitrate calculation method, ω i >0 is a preset parameter used to balance different machine vision tasks.

[0165] Implementation results:

[0166] In the above embodiment, the image compression model for machine vision is trained using the ImageNet dataset. In the first stage, λ is set to 0.0018, 0.0035, 0.0067, and 0.0130, respectively, corresponding to four different bitrate image compression models. The Adam optimizer is used for 2,000,000 optimization steps, and the initial learning rate for the first 1,800,000 steps is set to 10. -4 It then dropped to 10 -5 Finally, it dropped to 10 at 1,950,000 steps. -6 In the second stage, the COCO2017-train dataset was used for training. λ was set to 0.0018, 0.0035, 0.0067, and 0.0130, corresponding to four different bitrates. The Adam optimizer was used for 200,000 optimization steps, with the initial 50,000 steps having a learning rate of 10. -3 It was then reduced to 10 -4 .

[0167] In the above embodiments, image encoders from four trained machine vision-oriented image compression models are used to compress all images in the COCO2017-validation test set. The average bitrates of the compressed images are 0.0884 bpp, 0.1265 bpp, 0.177 bpp, and 0.2488 bpp, respectively. In these embodiments, image encoders from four trained machine vision-oriented image compression models are used to compress all images in the COCO2017-validation test set to obtain reconstructed images for machine vision tasks. The machine vision tasks selected in these embodiments are object detection based on Faster-RCNN and Mask-RCNN. For semantic segmentation tasks, the average bitrates for detection were 0.063 bpp, 0.093 bpp, 0.142 bpp, and 0.209 bpp, with average intersection-over-union (IoU) ratios of 28.887%, 32.225%, 35.456%, and 36.802%, respectively. For segmentation tasks, the average bitrates were 0.07 bpp, 0.103 bpp, 0.148 bpp, and 0.224 bpp, with average IoU ratios of 27.1%, 30.436%, 32.808%, and 34.147%, respectively.

[0168] Compared to methods using baseline neural networks, the image compression method for machine vision provided in the above embodiments of the present invention has better rate-distortion performance. Specifically, under the premise of the same bit rate overhead, it can improve the accuracy of reconstructed images for object detection tasks by 17.71% and for semantic segmentation tasks by 13.99%, thus improving the accuracy performance of rate-distortion tasks.

[0169] Based on the same inventive concept, in other embodiments of the present invention, a terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute the above-described image encoding method for machine and human vision, or image decoding method for machine and human vision, or image compression method for machine and human vision.

[0170] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the above-mentioned computer programs, computer instructions, etc., may be partitioned and stored in one or more memories.

[0171] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0172] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0173] Based on the same inventive concept, in other embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, can be used to perform the above-described image encoding method for machine and human vision, or, image decoding method for machine and human vision, or image compression method for machine and human vision.

[0174] Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a user device. Of course, the processor and storage medium can also exist as discrete components in a communication device.

[0175] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0176] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0177] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0178] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0179] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. An image coding method for machine and human vision, characterized in that, include: For the image to be compressed, a multi-channel feature map of the image is generated using an encoding convolutional neural network, which serves as the first feature map. The first feature map is quantized with a quantization step size of 1 to obtain a feature map oriented towards human visual encoding, which is used as the second feature map. For the first feature map, the super-prior information is extracted by the super-prior coding convolutional neural network and the super-prior information binary code stream is obtained. Then, the super-prior information binary code stream is decoded by the super-prior decoding convolutional neural network to generate reconstructed super-prior information. The quantization step size for machine vision coding is estimated using the reconstructed prior information and adjusted to an odd number. The first feature map is then quantized into quantized feature maps for N machine vision tasks based on the quantization step size, where N is a positive integer. Using the reconstructed prior information, the distribution of the quantized feature map and the distribution of the second feature map for the Nth to the 1st machine vision task are predicted sequentially using an entropy model; probability estimation is performed based on the predicted distribution, and arithmetic encoding is performed based on the estimated probability to obtain a layer-by-layer feature map binary code stream, which is then merged to obtain the feature map binary code stream. The binary code stream of the prior information and the binary code stream of the feature map are merged to obtain the compressed image binary code stream.

2. The image coding method for machine and human vision according to claim 1, characterized in that, For the first feature map, the process involves extracting prior information using a super-prior coding convolutional neural network and obtaining a binary code stream of the super-prior information. Subsequently, a super-prior decoding convolutional neural network is used to decode the binary code stream of the super-prior information to generate reconstructed super-prior information, including: The super-prior feature map is obtained from the first feature map by a super-prior coding convolutional neural network; The super-prior feature map is quantized to obtain a quantized super-prior feature map; The quantized super-prior feature map is subjected to a distribution estimation of mutually independent features, and the distribution estimation is used to perform arithmetic encoding on the quantized super-prior feature map to obtain a binary code stream of super-prior information. Arithmetic decoding is used to recover the prior information from the prior information binary code stream; Reconstructed hyperprior information is obtained from the recovered hyperprior information through a hyperprior decoding convolutional neural network; The super-prior decoding convolutional neural network is a deconvolutional neural network corresponding to the super-prior encoding convolutional neural network.

3. The image coding method for machine and human vision according to claim 1, characterized in that, The method involves estimating the quantization step size for machine vision coding using the reconstructed prior information and adjusting it to an odd number. Based on this quantization step size, the first feature map is quantized into quantized feature maps for N machine vision tasks, where N is a positive integer, including: Based on the information requirements of machine vision tasks, the indices i = 1, ..., N for N machine vision tasks are defined. The smaller the index, the greater the information required by the machine vision task. The machine vision tasks include image classification, object detection, semantic segmentation, and instance segmentation. The quantization step size relationship between N machine vision tasks is obtained by using the reconstructed prior information; Based on the correspondence between quantization step sizes of different machine vision tasks, the quantization step sizes for machine vision-oriented coding of different tasks are obtained. The first feature map is quantized using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks.

4. The image coding method for machine and human vision according to claim 3, characterized in that, The process of obtaining the quantization step size relationship between N machine vision tasks using the reconstructed prior information includes: For i = 1, ..., N, a quantized convolutional neural network is used to estimate the quantization step size relationship q of the first feature map between the i-th machine vision task and the (i+1)-th machine vision task. i Specifically: The reconstructed prior information Conv1 is a convolutional layer with a kernel size of 3×3. 3×3 A transformation is performed, using LeakyReLU as the nonlinear mapping layer, to obtain the latent quantization step size relationship information. The LeakyReLU function is as follows: Where α is a positive parameter used to control the gradient of negative values; The relationship information of the quantization step size of the subsurface Upsampling is performed by sequentially using the DeConv1 deconvolution layer of the SoftPlus nonlinear mapping layer. 3×3 and the deconvolutional layer DeConv2 using LeakyReLU nonlinear mapping layers 3×3 The rough quantization step size relationship is obtained. The SoftPlus function expression is as follows: SoftPlus(x)=log(1+e x ) The coarse quantization step size relationship Conv2 convolutional layer with a kernel size of 3×3 as input 3×3 Using a SoftPlus nonlinear mapping layer and adding a bias of 1, the quantization step size relationship is ensured to be no less than 1, thus obtaining a continuous quantization step size relationship. Adjust the relationship of the continuous quantization step size By approximating an odd number for the quantization step size, we obtain the quantization step size relationship q between the i-th machine vision task and the (i+1)-th machine vision task. i Specifically: in, Return the largest positive integer not greater than q.

5. The image coding method for machine and human vision according to claim 4, characterized in that, The step of obtaining the quantization step size for machine vision encoding for different tasks based on the correspondence between quantization step sizes for different machine vision tasks includes: For i = 1, ..., N, estimate the quantization step size Q of the first feature map under the i-th machine vision task. i :

6. The image coding method for machine and human vision according to claim 5, characterized in that, The process of quantizing the first feature map using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks is as follows: For i = 1, ..., N, obtain the quantized feature map corresponding to the i-th machine vision task. for: Where y is the first feature map, and Round(y) returns the result after rounding the decimal part of y.

7. The image coding method for machine and human vision according to claim 1, characterized in that, The process involves using reconstructed prior information to sequentially predict the distribution of the quantized feature maps and the distribution of the second feature map for the Nth to the 1st machine vision task using an entropy model, performing probability estimation based on the predicted distributions, performing arithmetic encoding based on the estimated probabilities, obtaining a layer-by-layer feature map binary code stream, and merging them to obtain the feature map binary code stream, including: Quantize the feature map of the Nth machine vision task The parameterized model is a Gaussian distribution, and the entropy model is used to predict the mean and variance of the Gaussian distribution using the reconstructed prior information; The quantized feature map of the Nth machine vision task is generated based on the predicted mean and variance of the Gaussian distribution. Probability estimation is performed, and arithmetic encoding is performed based on the estimated probability to obtain the binary code stream of the first layer feature map; The mean and variance of the Gaussian distribution of the quantized feature maps of the N-1th to the 1st machine vision task are predicted sequentially using the entropy model with the reconstructed prior information. Probability estimation is performed based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is performed based on the estimated probability to obtain the binary code stream of the feature maps of the 2nd to Nth layers. Using the reconstructed prior information, the mean and variance of the Gaussian distribution of the second feature map are predicted by the entropy model. Probability estimation is performed based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is performed based on the estimated probability to obtain the binary code stream of the N+1 layer feature map. Following the order of layers 1 to N+1, the binary code streams of the feature maps in layer N+1 are merged to obtain the binary code stream of the feature maps.

8. An image decoding method for machine and human vision, used to correspondingly decode the image encoding method for machine and human vision according to any one of claims 1-7, characterized in that, include: The image binary code stream is decomposed and compressed to obtain a priori information binary code stream and a feature map binary code stream; Decompose the binary code stream of the feature map to obtain an N+1 layer binary code stream of the feature map; Arithmetic decoding is used to recover the prior information from the binary code stream of the prior information; the deconvolutional neural network of the corresponding prior coding convolutional neural network is used as the prior decoding convolutional neural network to obtain the reconstructed prior information from the recovered prior information. A quantized convolutional neural network is used to estimate the quantization step size corresponding to N machine vision tasks from the reconstructed prior information and adjust it to an odd number. An entropy model is used to use the reconstructed prior information and combine it with the quantization step size of each machine vision task layer to predict the mean and variance of the Gaussian distribution of the quantized feature map of the Nth to the 1st machine vision task. According to the mean and variance of the Gaussian model, the binary code stream of the feature map is decoded layer by layer from the 1st to the Nth layer to obtain the quantized feature map of the corresponding reconstructed Nth to the 1st machine vision task. The mean and variance of the Gaussian distribution of the second feature map are predicted using the entropy model with the reconstructed prior information. The binary code stream of the N+1 layer feature map is then decoded based on the mean and variance of the Gaussian model to obtain the reconstructed second feature map. The reconstructed second feature map is then passed through a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain a reconstructed image for human vision. The reconstructed feature maps of N machine vision tasks are passed through the corresponding encoding convolutional neural network and the decoding convolutional neural network to obtain the reconstructed image for the machine vision task.

9. An image compression method for machine and human vision, characterized in that, include: Obtain the image to be compressed; The image to be compressed is compressed using the machine- and human-vision-oriented image coding method according to any one of claims 1-7 to obtain a compressed image binary bitstream; The image decoding method for machine and human vision described in claim 8 is used to decompress the compressed image binary stream to obtain a reconstructed image for human and machine vision.

10. The image compression method for machine and human vision according to claim 9, characterized in that, The parameters of the encoding convolutional neural network, decoding convolutional neural network, super-prior encoding convolutional neural network, super-prior decoding convolutional neural network, and entropy model are obtained through rate-distortion optimization. Among them, distortion x is the image to be compressed. This is a reconstructed image; λ>0 is a preset parameter used to balance distortion and bitrate; the bitrate R is based on the second feature map. The Gaussian distribution of each element is calculated as follows: in, It is the characteristic of the k-th element. It is the predicted distribution of the k-th element. To reconstruct the estimated distribution of the prior information, θ is used to estimate the number of codewords required for transmitting the reconstructed prior information. k ={φ k ,ψ k } is a set of learnable parameters.

11. The image compression method for machine and human vision according to claim 10, characterized in that, The parameters of the quantized convolutional neural networks for N machine vision tasks are obtained through optimization of bit rate and task accuracy: in, Let λ represent the loss function related to the accuracy of the i-th machine vision task. i >0 is a preset parameter used to balance the bitrate and task accuracy for the i-th machine vision task. i Let ω be the bitrate of the i-th machine vision task. i >0 is a preset parameter used to balance different machine vision tasks.

12. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to execute an image encoding method for machine and human vision as described in any one of claims 1-8, or an image decoding method for machine and human vision as described in claim 9, or an image compression method for machine and human vision as described in any one of claims 10-11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, this program can be used to perform an image encoding method for machine and human vision as described in any one of claims 1-8, or an image decoding method for machine and human vision as described in claim 9, or an image compression method for machine and human vision as described in any one of claims 10-11.

Images