Machine and human vision-oriented image coding and decoding method and compression method

Abstract

The invention provides an image coding and decoding method and a compression method for machine and human vision, and the method mainly comprises the steps: obtaining a first feature map and a second feature of a to-be-coded image, and obtaining the super-prior information of the first feature map; obtaining quantization step sizes of the to-be-coded image under different machine vision tasks, and performing quantization to obtain a corresponding feature map; modeling the feature map into Gaussian distribution, and predicting a corresponding mean value and variance through an entropy model; coding layer by layer according to the distribution and probability of each layer of feature map to obtain a feature map binary code stream; and combining the super-prior information binary code stream and the feature map binary code stream to obtain a compressed image binary code rate. According to the method, the quantization step size of feature map quantization is adaptively adjusted, transmission feature code streams for human vision and different machine vision tasks are formed, information in the feature map can be more efficiently utilized, the transmission code rate is reduced, meanwhile, the performance of the machine vision tasks is guaranteed, and meanwhile, the rate distortion performance oriented to human vision is balanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image encoding, decoding method and compression method oriented to machine and human vision. Background Art

[0002] In recent years, end-to-end image compression coding technology based on deep learning has developed rapidly. However, many image coding and transmission scenarios are not for human vision, but for machines to perform computer vision tasks on decoded images. Therefore, the main goal of the end-to-end image compression model is not only to maintain visual quality, but also to maintain the task accuracy of machine tasks at a given bit rate. In 2019, L. Duan published "Video coding for machines: A paradigm of collaborative compression and intelligent analytics" in the journal "IEEE Transactions on Image Processing" and proposed an end-to-end image compression coding scheme based on deep learning for machine vision. Y. Hu published "Towards coding for human and machine vision: A scalable image coding approach" at the 2020 "IEEE International Conference on Multimedia and Expo" and proposed an image compression coding framework that is simultaneously oriented to human vision and machine vision through scalable coding, which provides a wider range of application scenarios for image compression coding. In 2022, H. Choi proposed an image coding scheme that combines machine vision and human vision based on fixed feature channel grouping in "Scalable Image Coding for Humans and Machines" published in the journal "IEEE Transactions on Image Processing". The number of transmitted feature groups is selected according to the complexity of the machine task, which greatly improves the task accuracy of machine tasks at a given bit rate with little impact on human visual performance.

[0003] However, since the feature channel grouping is fixed, for different computer vision tasks, the specified feature channel group is not the optimal feature channel group, which results in redundancy in the transmitted code stream and suboptimal performance of the machine task. Summary of the invention

[0004] In view of the defects in the prior art, the object of the present invention is to provide an image encoding, decoding method and compression method oriented to machine and human vision.

[0005] A first aspect of the present invention provides an image coding method for machine and human vision, comprising:

[0006] For an image to be compressed, a coding convolutional neural network is used to generate a multi-channel feature map of the image as a first feature map; the first feature map is quantized according to a quantization step of 1 to obtain a feature map oriented to human visual coding as a second feature map;

[0007] For the first feature map, extracting super-prior information through a super-prior encoding convolutional neural network and obtaining a super-prior information binary code stream, and then decoding the super-prior information binary code stream through a super-prior decoding convolutional neural network to generate reconstructed super-prior information;

[0008] Using the reconstructed hyper-prior information, estimating a quantization step size for machine vision coding, and adjusting the step size to an odd number, and quantizing the first feature map into a quantized feature map for N machine vision tasks according to the quantization step size, where N is a positive integer;

[0009] The reconstructed hyper-prior information is used to sequentially use an entropy model to predict the distribution of the quantized feature map of the Nth machine vision task to the first machine vision task and the distribution of the second feature map; probability estimation is performed according to the predicted distribution, and arithmetic coding is performed according to the estimated probability to obtain layer-by-layer feature Figure 2 Binary code stream, merge to obtain features Figure 2 Binary code stream;

[0010] Merge the super-prior information binary code stream and the feature Figure 2 Binary code stream to get the compressed image binary code stream.

[0011] Preferably, for the first feature map, extracting super-prior information through a super-prior encoding convolutional neural network and obtaining a super-prior information binary code stream, and then decoding the super-prior information binary code stream through a super-prior decoding convolutional neural network to generate reconstructed super-prior information, including:

[0012] Obtaining a super prior feature map from the first feature map by a super prior encoding convolutional neural network;

[0013] quantizing the super a priori feature map to obtain a quantized super a priori feature map;

[0014] Performing distribution estimation of mutually independent features on the quantized super a priori feature map, and performing arithmetic coding on the quantized super a priori feature map using the distribution estimation to obtain a super a priori information binary code stream;

[0015] Recovering the super-a priori information from the super-a priori information binary code stream by using arithmetic decoding;

[0016] Reconstructing super-prior information from the recovered super-prior information by means of a super-prior decoding convolutional neural network;

[0017] Among them, the super priori decoding convolutional neural network is a deconvolutional neural network corresponding to the super priori encoding convolutional neural network.

[0018] Preferably, the method of using the reconstructed hyper-prior information to estimate a quantization step size for machine vision coding and adjusting it to an odd number, and quantizing the first feature map into a quantized feature map for N machine vision tasks according to the quantization step size, wherein N is a positive integer, includes:

[0019] According to the relationship between the amount of information required by the machine vision tasks, the indexes i=1,…,N of N machine vision tasks are specified, and the smaller the index, the greater the amount of information required by the machine vision task; the machine vision tasks include image classification, target detection, semantic segmentation and instance segmentation;

[0020] Using the reconstructed hyper-prior information, a quantization step length relationship between N machine vision tasks is obtained;

[0021] According to the corresponding relationship between the quantization step sizes of different machine vision tasks, the quantization step sizes for machine vision coding of different tasks are obtained;

[0022] The first feature map is quantized using the quantization step size for machine vision encoding to obtain quantized feature maps corresponding to N machine vision tasks.

[0023] Preferably, the using the reconstructed hyper-prior information to obtain the quantization step relationship between N machine vision tasks includes:

[0024] For i=1,…,N, use a quantized convolutional neural network to estimate the quantization step relationship q between the first feature map in the i-th machine vision task and the i+1-th machine vision task i , specifically:

[0025] The reconstructed hyper-prior information Convolutional layer Conv1 with input kernel size of 3×3 3×3 Transform and use LeakyReLU as the nonlinear mapping layer to obtain the latent quantization step relationship information

[0026]

[0027] Among them, the LeakyReLU function is:

[0028]

[0029] Among them, α is a positive parameter used to regulate the gradient of negative values;

[0030] Quantize the step length relationship information of the latent layer Upsampling is performed, and then the deconvolution layer DeConv1 using the SoftPlus nonlinear mapping layer is used. 3×3 And the deconvolution layer DeConv2 using the LeakyReLU nonlinear mapping layer 3×3 , and get the rough quantization step length relationship

[0031]

[0032] The SoftPlus function expression is:

[0033] SoftPlus(x)=log(1+e x )

[0034] The coarse quantization step length relationship Convolutional layer Conv2 with input kernel size of 3×3 3×3 , use the SoftPlus nonlinear mapping layer and add a bias of 1 to ensure that the relationship of the quantization step is not less than 1, and obtain the continuous quantization step relationship

[0035]

[0036] Adjust the continuous quantization step relationship Make the quantization step size approximate to an odd number, and obtain the quantization step size relationship q between the i-th machine vision task and the i+1-th machine vision task i , specifically:

[0037]

[0038] in, Returns the largest positive integer not greater than q.

[0039] Preferably, obtaining the quantization step sizes for machine vision coding of different tasks according to the corresponding relationship between the quantization step sizes of different machine vision tasks includes:

[0040] For i=1,…,N, estimate the quantization step size Q of the first feature map under the i-th machine vision task i :

[0041]

[0042] Preferably, the first feature map is quantized using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks, specifically:

[0043] For i=1,…,N, obtain the quantitative feature map corresponding to the i-th machine vision task for:

[0044]

[0045] Where y is the first feature map, and Round(y) returns the result after rounding the decimal part of y.

[0046] Preferably, the method uses the reconstructed hyper-prior information to sequentially use an entropy model to predict the distribution of the quantized feature map of the Nth machine vision task to the first machine vision task and the distribution of the second feature map, performs probability estimation according to the predicted distribution, performs arithmetic coding according to the estimated probability, and obtains the layer-by-layer feature map. Figure 2 Binary code stream, merge to obtain features Figure 2 Binary code stream, including:

[0047] The quantized feature map of the Nth machine vision task The parameterized model is a Gaussian distribution, and an entropy model is used to predict the mean and variance of the Gaussian distribution using the reconstructed hyper-prior information;

[0048] Quantitative feature map of the Nth machine vision task based on the predicted mean and variance of the Gaussian distribution Perform probability estimation, perform arithmetic coding based on the estimated probability, and obtain the first layer feature Figure 2 Binary code stream;

[0049] The entropy model is used in turn to predict the Gaussian distribution mean and variance of the quantized feature maps of the N-1th to the 1st machine vision tasks using the reconstructed super-prior information, and probability estimation is performed according to the predicted mean and variance of the Gaussian distribution, and arithmetic coding is performed according to the estimated probability to obtain the 2nd to Nth layer features. Figure 2 Binary code stream;

[0050] The entropy model is used to reconstruct the super prior information to predict the mean and variance of the Gaussian distribution of the second feature map, and probability estimation is performed according to the predicted mean and variance of the Gaussian distribution, and arithmetic coding is performed according to the estimated probability to obtain the N+1th layer feature map. Figure 2 Binary code stream;

[0051] Merge N+1 layer features in the order of layer 1 to layer N+1 Figure 2 Binary code stream, get the characteristics Figure 2 Binary code stream.

[0052] According to a second aspect of the present invention, there is provided a method for decoding an image oriented to machine vision and human vision, which is used for decoding any one of the methods for encoding an image oriented to machine vision and human vision, comprising:

[0053] Decompose and compress the image binary code stream to obtain the super prior information binary code stream and feature Figure 2 Binary code stream; decomposition of the features Figure 2 Binary code stream, obtain N+1 layer features Figure 2 Binary code stream;

[0054] Recovering the super-prior information from the super-prior information binary code stream by arithmetic decoding; using the deconvolution neural network corresponding to the super-prior encoding convolution neural network as the super-prior decoding convolution neural network to obtain the reconstructed super-prior information from the recovered super-prior information;

[0055] A quantized convolutional neural network is used to estimate the quantization step size corresponding to N machine vision tasks from the reconstructed super prior information and adjust it to an odd number. An entropy model is used to use the reconstructed super prior information and the quantization step size of each machine vision task layer to predict the mean and variance of the Gaussian distribution of the quantized feature map of the Nth to the first machine vision task one by one. The feature map is decoded layer by layer from the 1st layer to the Nth layer according to the mean and variance of the Gaussian model. Figure 2 The entropy model is used to predict the mean and variance of the Gaussian distribution of the second feature map using the reconstructed hyper-prior information, and the mean and variance of the N+1 layer feature map are calculated based on the Gaussian model. Figure 2 Decode the binary code stream to obtain a reconstructed second feature map;

[0056] Passing the reconstructed second feature map through a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain a reconstructed image for human vision;

[0057] The reconstructed feature maps of N machine vision tasks are passed through the decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain the reconstructed image for the machine vision task.

[0058] According to a third aspect of the present invention, there is provided an image compression method for machine and human vision, comprising:

[0059] Obtain the image to be compressed;

[0060] Compressing the image using any one of the machine and human vision-oriented image coding methods to obtain a compressed image binary code stream;

[0061] The image decoding method for machine and human vision is used to decompress the compressed image binary code stream to obtain a reconstructed image for human vision and machine vision.

[0062] Preferably, the parameters of the encoding convolutional neural network, the decoding convolutional neural network, the super a priori encoding convolutional neural network, the super a priori decoding convolutional neural network, and the entropy model are obtained by rate-distortion optimization:

[0063]

[0064] Among them, distortion x is the image to be compressed, is the reconstructed image, λ>0 is a preset parameter for balancing distortion and bit rate, and the bit rate R is calculated according to the second feature map The Gaussian distribution of each element is calculated as:

[0065]

[0066] in, is the feature of the kth element, is the predicted distribution of the kth element, is the estimated distribution of the reconstructed super-prior information, which is used to estimate the number of codewords required for the transmission of the reconstructed super-prior information, θ k ={φ k ,ψ k} is a set of learnable parameters.

[0067] Preferably, the parameters of the quantized convolutional neural network for N machine vision tasks are obtained by optimizing the bit rate and task accuracy:

[0068]

[0069] in, represents the loss function related to the accuracy of the i-th machine vision task, λ i >0 is the preset parameter used to balance the bit rate and task accuracy corresponding to the i-th machine vision task, R i is the bit rate of the i-th machine vision task, ω i >0 is the preset parameter used to balance between different machine vision tasks.

[0070] According to a fourth aspect of the present invention, a terminal comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the terminal can be used to execute any one of the image encoding methods for machine and human vision, or, any one of the image decoding methods for machine and human vision, or, any one of the image compression methods for machine and human vision.

[0071] According to a fifth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any one of the image encoding methods for machine and human vision, or, any one of the image decoding methods for machine and human vision, or, any one of the image compression methods for machine and human vision.

[0072] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0073] The image encoding, decoding and compression method for machine and human vision in the embodiment of the present invention adaptively adjusts the quantization step size of the feature map quantization to form a transmission feature code stream for human vision and different machine vision tasks, which can more efficiently utilize the information in the feature map, reduce the transmission bit rate, ensure the performance of machine vision tasks, and balance the rate-distortion performance for human vision.

[0074] The image encoding, decoding and compression method for machine and human vision in the embodiment of the present invention can enable a single code stream to perform a variety of different machine vision tasks and obtain performance similar to that of a single machine vision task.

[0075] The image encoding, decoding and compression method for machine and human vision in the embodiment of the present invention can generate a truncated single binary code stream through the quantization step size, form reconstructed images for different tasks of human vision and machine vision, adapt to terminals that perform different tasks, and do not need to switch different code stream versions for multiple different tasks, thereby improving user experience and reducing the overall bandwidth requirements of network transmission.

[0076] The image encoding, decoding and compression method for machine and human vision in the embodiment of the present invention can more efficiently utilize feature maps to perform downstream tasks of the image, and by controlling the quantization step size of the feature map, the transmission bit rate can be minimized on the basis of meeting the task performance indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0078] Figure 1 A flowchart of an image encoding method for machine and human vision according to an embodiment of the present invention;

[0079] Figure 2 A flowchart of an image encoding method for machine and human vision in a preferred embodiment of the present invention;

[0080] Figure 3A flowchart of an image decoding method for machine and human vision in a preferred embodiment of the present invention;

[0081] Figure 4 A flowchart of an image compression method for machine and human vision in a preferred embodiment of the present invention;

[0082] Figure 5 The present invention is a flowchart of a method for training an image compression model for machine and human vision in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0083] The present invention is described in detail below in conjunction with specific embodiments and accompanying drawings. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0084] Reference Figure 1 FIG. 1 is a flowchart of image coding for machine vision and human vision in an embodiment of the present invention. This embodiment provides an image coding method for machine vision, including steps S100-S500.

[0085] Specific:

[0086] S100. For the image to be compressed, using a coding convolutional neural network to generate a multi-channel feature map of the image as a first feature map; quantizing the first feature map according to a quantization step of 1 to obtain a feature map for human visual coding as a second feature map;

[0087] S200. For the first feature map obtained in S100, extract the super-prior information through the super-prior coding convolutional neural network, obtain the super-prior information binary code stream, and decode the super-prior information binary code stream to generate reconstructed super-prior information;

[0088] S300. Use the reconstructed hyper-prior information obtained in S200 to estimate the quantization step size for machine vision coding, and adjust it to an odd number to meet the embedded quantization condition, eliminate codeword redundancy, and improve codeword utilization. According to the quantization step size, the first feature map obtained in S100 is quantized into a quantized feature map for N machine vision tasks, where N is a positive integer;

[0089] S400. Using the reconstructed hyper-prior information obtained in S200, the entropy model is used to predict the distribution of the quantized feature map and the second feature map of the Nth machine vision task to the first machine vision task in turn; probability estimation is performed according to the predicted distribution, and arithmetic coding is performed according to the estimated probability to obtain layer-by-layer feature maps. Figure 2 Binary code stream, merge to obtain features Figure 2 Binary code stream;

[0090] S500. Combine the super-prior information binary code stream obtained in S200 and the features obtained in S400 Figure 2 Binary code stream to get the compressed image binary code stream.

[0091] In step S100 of the above embodiment, the first feature map mentioned is a three-dimensional feature map of the image to be compressed, and the dimensions of the three-dimensional features include feature channels, feature heights, and feature widths. In S400, it is mentioned that the entropy model network is implemented using existing technologies such as super-prior networks and spatial context models. For example, a super-prior feature map is recovered from super-prior information using a super-prior decoding convolutional neural network, corresponding to the mean and variance of the Gaussian distribution. When encoding the feature map and the second feature map corresponding to each machine vision task, a progressive coding scheme is used to obtain a layered code stream, and the progressive coding scheme is implemented using existing technologies.

[0092] The above-mentioned embodiments of the present invention can efficiently extract features and information that are more needed by machine vision, while reducing redundant feature transmission for machine vision tasks, and can reduce image compression bit rate overhead while effectively improving the accuracy of downstream tasks. At the same time, the idea of ​​progressive coding is used, and the bit stream obtained by coding can be layered. A single bit stream can perform multiple machine vision tasks at the same time, which has higher efficiency and reduces the bit stream transmission overhead under multiple tasks.

[0093] Figure 2 Flow chart of a method for image coding for machine and human vision in a preferred embodiment of the present invention. In this preferred embodiment, the image coding for machine vision may include relatively detailed steps S101-S109, specifically:

[0094] S101, 3D feature extraction: the image to be compressed is encoded with a feature map obtained by a convolutional neural network to obtain a 3D feature, i.e., a first feature map;

[0095] S102, extracting and quantizing super-prior information: using a super-prior coding convolutional neural network to obtain super-prior information on the first feature map; rounding off the super-prior information and quantizing it to an integer;

[0096] S103, super a priori information encoding: establishing a distribution estimate between each feature of the super a priori information that is independent of each other, and using the distribution estimate to perform arithmetic encoding on the super a priori information to obtain a super a priori information binary code stream;

[0097] S104, super-prior information decoding: performing arithmetic decoding on the super-prior information binary code stream to obtain a restored super-prior information feature; using a super-prior decoding convolutional neural network to obtain a super-prior feature map on the restored super-prior information;

[0098] S105, quantization: the first feature Figure 4 Round off and quantize to an integer to obtain a second feature map;

[0099] S106, machine vision task quantization: based on the recovered super prior information, the quantization step relationship between different task layers is obtained by quantizing the convolutional neural network; then, based on the obtained corresponding relationship between the quantization step sizes of different task layers, the quantization step sizes under different tasks are obtained; finally, the first feature map is quantized using the quantization step size for machine vision encoding to obtain feature maps corresponding to different machine vision tasks.

[0100] S107, Probabilistic Entropy Modeling: Parameterize the feature distribution of each feature map into a Gaussian distribution, and use the hyper-prior information and context model to predict the mean and variance of the corresponding Gaussian distribution;

[0101] S108, progressive coding of three-dimensional features: Probabilistically estimate the features of all feature maps based on the mean and variance of the predicted Gaussian distribution of all feature maps, use the existing progressive coding technology to encode all feature maps layer by layer, and merge them to obtain a feature binary code stream of the three-dimensional features.

[0102] S109, further merging the feature binary code stream with the super a priori feature binary code stream to obtain a compressed image binary code stream.

[0103] In order to ensure that each step obtains the best execution effect, in some embodiments of the present invention, preferred solutions of S200, S300, and S400 are provided respectively.

[0104] In one embodiment of the present invention, a preferred solution for implementing step S200 is provided, and the specific steps are as follows:

[0105] S21, obtaining a super prior feature map from the first feature map through a super prior encoding convolutional neural network;

[0106] S22, quantizing the super prior feature map obtained in S21 to obtain a quantized super prior feature map;

[0107] S23, performing distribution estimation of mutually independent features on the quantized super a priori feature map obtained in S22, and performing arithmetic coding on the quantized super a priori feature map using the distribution estimation to obtain a super a priori information binary code stream;

[0108] S24, using arithmetic decoding to recover the super-prior information from the super-prior information binary code stream obtained in S23;

[0109] S25, reconstructing super-prior information from the super-prior information recovered from S24 through a super-prior decoding convolutional neural network;

[0110] Among them, the super prior decoding convolutional neural network is a deconvolutional neural network corresponding to the super prior encoding convolutional neural network.

[0111] In the above embodiment, steps S21-S25 realize the encoding of the prior information of the first and second feature maps to be encoded, reduce the encoding overhead of the first and second feature maps, and improve the overall encoding efficiency.

[0112] In one embodiment of the present invention, a preferred method for implementing step S300 is provided, and the specific steps are as follows:

[0113] S31, according to the relationship between the amount of information required by the machine vision tasks, specify the indexes i=1,…,N of N machine vision tasks, wherein the smaller the index, the greater the amount of information required by the machine vision task; the machine vision tasks include image classification, object detection, semantic segmentation, and instance segmentation;

[0114] S32, using the reconstructed hyper-prior information determined in S200 to obtain a quantization step length relationship between N machine vision tasks;

[0115] S33, according to the corresponding relationship between the quantization step sizes of different machine vision tasks, obtaining the quantization step sizes for machine vision coding of different tasks;

[0116] S34, quantizing the first feature map using the quantization step size for machine vision coding obtained in S33 to obtain quantized feature maps corresponding to N machine vision tasks.

[0117] In the above embodiment, the multi-feature map quantization implemented in steps S31-S34 can realize the transmission of the content required for completing multiple visual tasks with a single bit stream, thereby improving the application scope of the above method.

[0118] In a preferred embodiment, S32 is implemented to obtain the quantization step length relationship between N machine vision tasks by reconstructing the hyper-prior information, including:

[0119] For i=1,…,N, use a quantized convolutional neural network to estimate the quantization step relationship q between the first feature map in the i-th machine vision task and the i+1-th machine vision task. i , specifically:

[0120] Reconstructing the super prior information Convolutional layer Conv1 with input kernel size of 3×3 3×3 Transform and use LeakyReLU as the nonlinear mapping layer to obtain the latent quantization step relationship information

[0121]

[0122] Among them, the LeakyReLU function is:

[0123]

[0124] Among them, α is a positive parameter used to regulate the gradient of negative values;

[0125] Quantize the step length relationship information of the latent layer Upsampling is performed, and then the deconvolution layer DeConv1 using the SoftPlus nonlinear mapping layer is used. 3×3 And the deconvolution layer DeConv2 using the LeakyReLU nonlinear mapping layer 3×3 , and get the rough quantization step length relationship

[0126]

[0127] The SoftPlus function expression is as follows:

[0128] SoftPlus(x)=log(1+e x )

[0129] The coarse quantization step length relationship Convolutional layer Conv2 with input kernel size of 3×3 3×3 , use the SoftPlus nonlinear mapping layer and add a bias of 1 to ensure that the relationship of the quantization step is not less than 1, and obtain the continuous quantization step relationship

[0130]

[0131] Adjust the relationship between consecutive quantization steps Make the quantization step size approximately an odd number, satisfy the embedded quantization condition, eliminate codeword redundancy, improve codeword utilization, and obtain the quantization step size relationship q between the i-th machine vision task and the i+1-th machine vision task i , the formula is:

[0132]

[0133] in, Returns the largest positive integer not greater than q.

[0134] In a preferred embodiment, S33 is implemented to obtain quantization step sizes for machine vision coding of different tasks according to the corresponding relationship between quantization step sizes of different machine vision tasks, including:

[0135] For i=1,…,N, estimate the quantization step size Q of the first feature map under the i-th machine vision task i :

[0136]

[0137] In a preferred embodiment, S34 is implemented to quantize the first feature map using a quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks, specifically:

[0138] For i=1,…,N, obtain the quantitative feature map corresponding to the i-th machine vision task for:

[0139]

[0140] Where y is the first feature map, and Round(y) returns the result after rounding the decimal part of y.

[0141] In the above-mentioned embodiment of the present invention, by adjusting the quantization step size of the feature map under the machine vision task, the present invention can efficiently retain the information that is more needed by the machine vision, reduce the redundant information for the machine vision, reduce the image compression bit rate overhead, and effectively improve the accuracy of downstream tasks.

[0142] In a preferred embodiment of the present invention, S400 is implemented, and the specific process is as follows:

[0143] S41, the quantized feature map of the Nth machine vision task The parameterized model is a Gaussian distribution, and the entropy model is used to predict the mean and variance of the Gaussian distribution using the reconstructed hyper-prior information;

[0144] Quantitative feature map of the Nth machine vision task based on the mean and variance of the predicted Gaussian distribution Perform probability estimation, perform arithmetic coding based on the estimated probability, and obtain the first layer features Figure 2 Binary code stream;

[0145] S42, sequentially using the entropy model to reconstruct the hyper-prior information to predict the Gaussian distribution mean and variance of the quantized feature maps of the N-1th to the 1st machine vision tasks, performing probability estimation based on the predicted mean and variance of the Gaussian distribution, performing arithmetic coding based on the estimated probability, and obtaining the 2nd to Nth layer features Figure 2 Binary code stream;

[0146] S43, using the entropy model to reconstruct the super prior information, predict the mean and variance of the Gaussian distribution of the second feature map, perform probability estimation based on the predicted mean and variance of the Gaussian distribution, perform arithmetic coding based on the estimated probability, and obtain the N+1th layer feature map Figure 2 Binary code stream;

[0147] S44, merge N+1 layer features in the order of layer 1 to layer N+1 Figure 2 Binary code stream, get the characteristics Figure 2 Binary code stream.

[0148] The above-mentioned embodiment of the present invention uses the idea of ​​progressive coding, and the bit stream obtained by coding can be layered. A single bit stream can perform multiple machine vision tasks at the same time, has higher efficiency, and reduces the bit stream transmission overhead under multi-tasking.

[0149] Based on the same inventive concept, another embodiment of the present invention provides an image decoding method for machine and human vision, as shown in the flowchart: Figure 3 As shown, including:

[0150] S201, code stream decomposition: decompose the compressed image binary code stream to obtain features Figure 2 Binary code stream and super-prior information binary code stream; decomposition characteristics Figure 2 Binary code stream, obtain N+1 layer features Figure 2 Binary code stream;

[0151] S202, super a priori information decoding: performing arithmetic decoding on the super a priori information binary code stream to obtain restored super a priori information; using a super a priori decoding convolutional neural network on the restored super a priori information to obtain reconstructed super a priori information;

[0152] S203, Probabilistic Entropy Modeling: The structure and parameter values ​​of this module are similar to Figure 2 The probability entropy modeling in the illustrated embodiment is exactly the same; the entropy model is used to reconstruct the hyper-prior information to predict the mean and variance of the Gaussian distribution of the quantized feature map of the Nth to the first machine vision task one by one, and the features are decoded layer by layer from the 1st layer to the Nth layer according to the mean and variance of the Gaussian model. Figure 2 The quantized feature maps corresponding to the Nth to the first machine vision tasks are obtained by using the entropy model to reconstruct the hyper-prior information to predict the mean and variance of the Gaussian distribution of the second feature map, and the mean and variance of the N+1th layer feature map are predicted according to the Gaussian model. Figure 2 Decode the binary code stream to obtain a second feature map;

[0153] S204, prediction of quantization step size of machine vision tasks: The structure of this module is exactly the same as the part of calculating quantization step size in S7, and predicts the quantization step size of feature maps corresponding to each machine vision task from the restored reconstructed super prior information. Specifically, a quantized convolutional neural network is used to estimate the quantization step size corresponding to N machine vision tasks from the reconstructed super prior information, and the quantization step size is adjusted to an odd number.

[0154] S205, progressive decoding of three-dimensional features: according to the mean and variance of the Gaussian model in S203 and the quantization step size of the feature map corresponding to the machine vision task in S204, the feature Figure 2The binary code stream is progressively decoded, and the progressive decoding adopts the existing technology to obtain the restored feature map corresponding to the machine vision task and the second feature map corresponding to the human vision layer by layer.

[0155] S206, image reconstruction: the second feature map reconstructed in S205 is passed through a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain a reconstructed image for human vision;

[0156] The N machine vision task reconstructed feature maps obtained in S205 are decoded through the corresponding encoding convolutional neural network to obtain a reconstructed image for the machine vision task.

[0157] The image decoding method based on adaptive quantization step size for machine vision combined with human vision in this embodiment is used for Figure 2 The image coding method for machine vision shown in FIG. Figure 2 The corresponding description in the illustrated embodiment will not be repeated here.

[0158] According to the same inventive concept, in another embodiment of the present invention, a method for image compression for machine and human vision is provided, and its main steps are as follows:

[0159] The first step is to obtain the image to be compressed;

[0160] The second step is to compress the image using an image coding method for machine and human vision in the above embodiment to obtain a compressed image binary code stream;

[0161] The third step is to use the image decoding method for machine and human vision in the above embodiment to decompress the compressed image binary code stream to obtain a reconstructed image for human vision and machine vision.

[0162] Furthermore, in a specific embodiment, a method for image compression oriented to machine and human vision is used to compress all images in the COCO2017 test set. The flowchart of the method is as follows: Figure 4 As shown, including:

[0163] S301, using an extraction module: sequentially obtaining images to be compressed;

[0164] S302, using an encoder: encoding the image to be compressed according to the machine vision-oriented image encoding method in the above embodiment to obtain a compressed image binary code stream.

[0165] Specifically, the number of channels of the extracted three-dimensional features is 192, and the downsampling multiple is 16; the number of machine vision tasks set is 2, namely the target detection task and the semantic segmentation task.

[0166] S303, using a decoder: decompressing the compressed image binary code stream according to an image decoding method for machine vision to obtain a partially reconstructed image for two types of downstream image tasks and a reconstructed image for human vision.

[0167] The compression method in the above embodiment of the present invention is based on Figure 1 , 2 Image coding methods for machine and human vision and Figure 3 The image decoding method for machine and human vision shown is implemented. For the parts not described in detail above, reference can be made to the corresponding description in the above embodiments and will not be repeated here.

[0168] The compression method in the above embodiment of the present invention can efficiently extract the information that is more needed by machine vision, while reducing the redundant information for machine vision, and can reduce the image compression bit rate overhead, effectively improving the accuracy of downstream tasks. At the same time, the code stream obtained by progressive coding can be layered, and a single code stream can perform multiple machine vision tasks at the same time, which has higher efficiency and reduces the code stream transmission overhead under multiple tasks.

[0169] For the above-mentioned image compression method, the compression effect can be improved by training the model. For example, the training set consists of all images in the COCO2017-train data, and the trained model can be used to implement image encoding and decoding for machine vision. Specifically, Figure 5 As shown, in a preferred embodiment, the image compression method includes:

[0170] S401, 3D feature extraction: randomly crop the images in the training set into images with a resolution of 256×256, and input them into the encoding convolutional neural network to obtain 3D features. The encoding convolutional neural network is composed of three residual blocks, one attention mechanism module, three residual blocks, one convolution layer and one attention mechanism module in cascade sequence. The decoding neural network here can also be other network structures, such as four-layer convolution cascade; the number of 3D feature channels is 128 or 192, and the height and width are both 16; of course, the above-mentioned image resolution size and 3D feature parameters in this embodiment can be adjusted according to actual conditions, and are not limited to those described in this step, and the parameters in the following steps are also the same.

[0171] S402, super-prior information extraction and noise addition: input the three-dimensional features into the super-prior coding convolutional neural network to obtain super-prior features. In order to avoid the gradient problem caused by the quantization operation, a uniformly distributed noise of -0.5 to 0.5 is added to the super-prior features to simulate the actual quantization, wherein: the super-prior coding convolutional neural network is composed of three cascaded convolutional layers; of course, the super-prior coding neural network can also be other network structures, such as three cascaded residual modules, etc., and is not limited to the second convolutional neural network in this step.

[0172] S403, super a priori bit rate estimation: establishing a distribution estimation between each feature that is independent of each other for the super a priori information, and using the distribution estimation to obtain a super a priori bit rate estimation;

[0173] S404, obtaining super-prior information: inputting the super-prior information after adding noise into the super-prior decoding convolutional neural network to obtain super-prior information, wherein: the super-prior decoding convolutional neural network is composed of three convolutional layers in cascade;

[0174] S405, machine vision quantization step prediction: input the super prior information after adding noise into the quantized convolutional neural network to obtain the quantization step relationship of the feature map under each machine vision task, and further obtain the quantization step of the feature map under each machine vision task, and quantize the result of the quantized convolutional neural network to an odd number so that the quantization step of each layer satisfies the relationship of odd multiples. Here, in order to avoid the gradient problem caused by the quantization operation, the gradient is set to 1 during the quantization operation;

[0175] S406, feature noise addition: To avoid the gradient problem caused by the quantization operation, a uniformly distributed noise of -0.5Q to 0.5Q is added to the three-dimensional feature according to the corresponding quantization step Q of the feature map to simulate the actual quantization; if it is human vision, Q=1;

[0176] S407, Gaussian distribution probability modeling: The structure of this module is Figure 2 The probability entropy modeling in the illustrated embodiment is exactly the same; the mean and variance of the Gaussian distribution of each group are predicted using super-prior information and an autoregressive model based on spatial context.

[0177] S408, characteristic code rate estimation: obtaining characteristic code rate estimation according to the predicted mean and variance of the Gaussian distribution;

[0178] S409, total bit rate estimation: adding the super a priori characteristic bit rate estimation and the characteristic bit rate estimation to obtain a total bit rate estimation;

[0179] S410, image reconstruction: input the noisy three-dimensional features into the decoding convolutional neural network to obtain a reconstructed image. The decoding convolutional neural network is composed of an attention mechanism module, three residual blocks, an attention mechanism module, three residual blocks and an upsampling convolutional layer, which are cascaded in sequence. The decoding convolutional neural network here can also be other network structures, corresponding to the encoding convolutional neural network; the above-mentioned image compression method for machine vision also needs to meet the image compression model that serves human vision, and can be further optimized in stages using the stochastic gradient descent method. In the first stage, the machine vision quantization step prediction module is disabled, and the quantization step is set to Q=1. Only the encoding convolutional neural network, the decoding convolutional neural network, the super prior encoding convolutional neural network, the super prior decoding convolutional neural network and the entropy model are optimized. The loss function of the first stage is:

[0180]

[0181] Among them, distortion x is the image to be compressed, is the reconstructed image, λ>0 is a preset parameter for balancing distortion and bit rate, and the bit rate R is calculated according to the second feature map The Gaussian distribution of each element is calculated as:

[0182]

[0183] in, is the feature of the kth element, is the predicted distribution of the kth element, θ k ={φ k ,ψ k} is a set of learnable parameters.

[0184] In the second stage, the encoding convolutional neural network, decoding convolutional neural network, super prior encoding convolutional neural network, super prior decoding convolutional neural network and entropy model are fixed, and the machine vision quantization step prediction module is optimized, and its parameters are optimized by the following formula:

[0185]

[0186] in, represents the loss function related to the accuracy of the i-th machine vision task, λ i >0 is a preset parameter used to balance the bit rate and task accuracy corresponding to the i-th machine vision task. The bit rate R of the i-th machine vision task i The bit rate calculation method is consistent with the first stage, ω i >0 is the preset parameter used to balance between different machine vision tasks.

[0187] Implementation effect:

[0188] In the above embodiment, the ImageNet dataset is used to train the image compression model for machine vision. In the first stage, λ is set to 0.0018, 0.0035, 0.0067, and 0.0130, respectively, corresponding to four image compression models with different bit rates. The Adam optimizer is used for 2,000,000 steps of optimization. The initial 1,800,000 steps of learning rate is set to 10 -4 , then dropped to 10 -5 , and finally dropped to 10 at 1950000 steps -6 In the second stage, the COCO2017-train dataset was used for training, and λ was set to 0.0018, 0.0035, 0.0067, and 0.0130, corresponding to four different bit rates, respectively. The Adam optimizer was used for 200,000 steps of optimization, and the initial 50,000 steps of learning rate was set to 10 -3 , then reduced to 10 -4 .

[0189] In the above embodiment, the image encoders of the four machine vision-oriented image compression models are used to compress all images in the COCO2017-validation test set. The average bit rates of the compressed images are 0.0884bpp, 0.1265bpp, 0.177bpp, and 0.2488bpp, respectively; In the above embodiment, the image encoders of the four machine vision-oriented image compression models are used to compress all images in the COCO2017-validation test set to obtain reconstructed images for machine vision tasks. The machine vision tasks selected in the embodiment are target detection tasks based on Faster-RCNN and Mask-RCNN. The average bit rates of semantic segmentation and detection tasks are 0.063bpp, 0.093bpp, 0.142bpp, and 0.209bpp, respectively, and the average intersection-over-union ratios of detection are 28.887%, 32.225%, 35.456%, and 36.802%, respectively; the average bit rates of segmentation tasks are 0.07bpp, 0.103bpp, 0.148bpp, and 0.224bpp, respectively, and the average intersection-over-union ratios of segmentation are 27.1%, 30.436%, 32.808%, and 34.147%, respectively.

[0190] Compared with the method using the benchmark neural network, the image compression method for machine vision provided in the above embodiment of the present invention has better rate-distortion performance. Specifically, under the premise of the same bit rate overhead, the accuracy of the reconstructed image for target detection tasks can be improved by 17.71% and the accuracy of the semantic segmentation task can be improved by 13.99%, thereby improving the rate-task accuracy performance.

[0191] Based on the same inventive concept, in other embodiments of the present invention, a terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute the above-mentioned image encoding method for machine and human vision, or, the image decoding method for machine and human vision, or, the image compression method for machine and human vision.

[0192] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above-mentioned computer programs, computer instructions, etc. can be partitioned and stored in one or more memories.

[0193] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0194] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0195] Based on the same inventive concept, in other embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it can be used to execute the above-mentioned image encoding method for machine and human vision, or, the image decoding method for machine and human vision, or, the image compression method for machine and human vision.

[0196] Among them, computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.

[0197] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0199] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0200] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0201] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. An image coding method for machine and human vision, characterized in that: include: For an image to be compressed, using an encoding convolutional neural network to generate a multi-channel feature map of the image as a first feature map; quantizing the first feature map according to a quantization step of 1 to obtain a feature map for human visual coding as a second feature map; For the first feature map, extracting super-prior information through a super-prior encoding convolutional neural network and obtaining a super-prior information binary code stream, and then decoding the super-prior information binary code stream through a super-prior decoding convolutional neural network to generate reconstructed super-prior information; Using the reconstructed hyper-prior information, estimating a quantization step size for machine vision coding, and adjusting the step size to an odd number, and quantizing the first feature map into a quantized feature map for N machine vision tasks according to the quantization step size, where N is a positive integer; The reconstructed hyper-prior information is used to sequentially use an entropy model to predict the distribution of the quantized feature maps of the Nth machine vision task to the first machine vision task and the distribution of the second feature map; probability estimation is performed according to the predicted distribution, arithmetic coding is performed according to the estimated probability, and a binary code stream of the feature maps layer by layer is obtained, and the binary code stream of the feature maps is obtained by merging; The super a priori information binary code stream and the feature map binary code stream are combined to obtain a compressed image binary code stream.

2. The image coding method for machine and human vision according to claim 1, characterized in that: For the first feature map, extracting super-prior information through a super-prior encoding convolutional neural network and obtaining a super-prior information binary code stream, and then decoding the super-prior information binary code stream through a super-prior decoding convolutional neural network to generate reconstructed super-prior information, including: Obtaining a super prior feature map from the first feature map by a super prior encoding convolutional neural network; quantizing the super a priori feature map to obtain a quantized super a priori feature map; Performing distribution estimation of mutually independent features on the quantized super a priori feature map, and performing arithmetic coding on the quantized super a priori feature map using the distribution estimation to obtain a super a priori information binary code stream; Recovering the super-a priori information from the super-a priori information binary code stream by using arithmetic decoding; Obtaining reconstructed super-prior information from the recovered super-prior information by means of a super-prior decoding convolutional neural network; Among them, the super priori decoding convolutional neural network is a deconvolutional neural network corresponding to the super priori encoding convolutional neural network.

3. The image coding method for machine and human vision according to claim 1, characterized in that: The method of estimating a quantization step size for machine vision coding by using the reconstructed hyper-prior information and adjusting the step size to an odd number, and quantizing the first feature map into a quantized feature map for N machine vision tasks according to the quantization step size, wherein N is a positive integer, includes: According to the relationship between the amount of information required by the machine vision tasks, the indexes i=1,…,N of N machine vision tasks are specified, and the smaller the index, the greater the amount of information required by the machine vision task; the machine vision tasks include image classification, target detection, semantic segmentation and instance segmentation; Using the reconstructed hyper-prior information, a quantization step length relationship between N machine vision tasks is obtained; According to the corresponding relationship between the quantization step sizes of different machine vision tasks, the quantization step sizes for machine vision coding of different tasks are obtained; The first feature map is quantized using the quantization step size for machine vision encoding to obtain quantized feature maps corresponding to N machine vision tasks.

4. The image coding method for machine and human vision according to claim 3, characterized in that: The method of obtaining the quantization step length relationship between N machine vision tasks by using the reconstructed hyper-prior information includes: For i=1,…,N, use a quantized convolutional neural network to estimate the quantization step relationship q between the first feature map in the i-th machine vision task and the i+1-th machine vision task i , specifically: The reconstructed hyper-prior information Convolutional layer Conv1 with input kernel size of 3×3 3×3 Transform and use LeakyReLU as the nonlinear mapping layer to obtain the latent quantization step relationship information Among them, the LeakyReLU function is: Among them, α is a positive parameter used to regulate the gradient of negative values; Quantize the step length relationship information of the latent layer Upsampling is performed, and then the deconvolution layer DeConv1 using the SoftPlus nonlinear mapping layer is used. 3×3 And the deconvolution layer DeConv2 using the LeakyReLU nonlinear mapping layer 3×3 , and get the rough quantization step length relationship The SoftPlus function expression is: SoftPlus(x)=log(1+e x ) The coarse quantization step length relationship Convolutional layer Conv2 with input kernel size of 3×3 3×3 , use the SoftPlus nonlinear mapping layer and add a bias of 1 to ensure that the relationship of the quantization step is not less than 1, and obtain the continuous quantization step relationship Adjust the continuous quantization step relationship Make the quantization step size approximately an odd number, and obtain the quantization step size relationship q between the oth machine vision task and the o+1th machine vision task i , specifically: in, Returns the largest positive integer not greater than q.

5. The image coding method for machine and human vision according to claim 4, characterized in that: The method of obtaining the quantization step sizes for machine vision coding of different tasks according to the corresponding relationship between the quantization step sizes of different machine vision tasks includes: For i=1,…,N, estimate the quantization step size Q of the first feature map under the i-th machine vision task i :

6. The image coding method for machine and human vision according to claim 5, characterized in that: The first feature map is quantized by using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks, specifically: For i=1,…,N, obtain the quantitative feature map corresponding to the i-th machine vision task for: Where y is the first feature map, and Round(y) returns the result after rounding the decimal part of y.

7. The image coding method for machine and human vision according to claim 1, characterized in that: The method uses the reconstructed hyper-prior information to sequentially use an entropy model to predict the distribution of the quantized feature maps of the Nth machine vision task to the first machine vision task and the distribution of the second feature map, performs probability estimation according to the predicted distribution, performs arithmetic coding according to the estimated probability, obtains a binary code stream of the feature maps layer by layer, and combines to obtain a binary code stream of the feature maps, including: The quantized feature map of the Nth machine vision task The parameterized model is a Gaussian distribution, and an entropy model is used to predict the mean and variance of the Gaussian distribution using the reconstructed hyper-prior information; Quantitative feature map of the Nth machine vision task based on the predicted mean and variance of the Gaussian distribution Performing probability estimation, and performing arithmetic coding according to the estimated probability to obtain a binary code stream of the first layer feature map; The entropy model is used in sequence to predict the mean and variance of the Gaussian distribution of the quantized feature maps of the N-1th to the 1st machine vision tasks using the reconstructed super-prior information, probability estimation is performed according to the predicted mean and variance of the Gaussian distribution, and arithmetic coding is performed according to the estimated probability to obtain a binary code stream of the feature maps of the 2nd to the Nth layers; Adopting an entropy model and utilizing the reconstructed super-prior information to predict the mean and variance of the Gaussian distribution of the second feature map, performing probability estimation according to the predicted mean and variance of the Gaussian distribution, performing arithmetic coding according to the estimated probability, and obtaining a binary code stream of the feature map of the N+1th layer; In the order of layer 1 to layer N+1, the feature map binary code streams of layer N+1 are merged to obtain the feature map binary code stream.

8. An image decoding method for machine and human vision, used for correspondingly decoding the image encoding method for machine and human vision according to any one of claims 1 to 7, characterized in that: include: Decomposing and compressing the image binary code stream to obtain a super a priori information binary code stream and a feature map binary code stream; Decomposing the feature map binary code stream to obtain an N+1 layer feature map binary code stream; Recovering the super-prior information from the super-prior information binary code stream by arithmetic decoding; using the deconvolution neural network corresponding to the super-prior encoding convolution neural network as the super-prior decoding convolution neural network to obtain the reconstructed super-prior information from the recovered super-prior information; A quantized convolutional neural network is used to estimate the quantization step sizes corresponding to N machine vision tasks from the reconstructed super-prior information and adjust them to an odd number. An entropy model is used to utilize the reconstructed super-prior information and to predict the mean and variance of the Gaussian distribution of the quantized feature maps of the Nth to the first machine vision tasks one by one in combination with the quantization step sizes of each machine vision task layer. The feature map binary code stream is decoded layer by layer from the first layer to the Nth layer according to the mean and variance of the Gaussian model to obtain the corresponding reconstructed quantized feature maps of the Nth to the first machine vision tasks. The entropy model is used to predict the mean and variance of the Gaussian distribution of the second feature map using the reconstructed super-prior information, and the binary code stream of the feature map of the N+1th layer is decoded according to the mean and variance of the Gaussian model to obtain a reconstructed second feature map; Passing the reconstructed second feature map through a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain a reconstructed image for human vision; The reconstructed feature maps of N machine vision tasks are passed through the decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain the reconstructed image for the machine vision task.

9. An image compression method for machine and human vision, characterized in that: include: Obtain the image to be compressed; Compressing the image to be compressed using the machine and human vision-oriented image coding method according to any one of claims 1 to 7 to obtain a compressed image binary code stream; The image decoding method for machine and human vision as described in claim 8 is used to decompress the compressed image binary code stream to obtain a reconstructed image for human vision and machine vision.

10. The image compression method for machine and human vision according to claim 9, characterized in that: The parameters of the encoding convolutional neural network, decoding convolutional neural network, super prior encoding convolutional neural network, super prior decoding convolutional neural network, and entropy model are obtained through rate distortion optimization: Among them, distortion x is the image to be compressed, is the reconstructed image, λ>0 is a preset parameter for balancing distortion and bit rate, and the bit rate R is calculated according to the second feature map The Gaussian distribution of each element is calculated as: in, is the feature of the kth element, is the predicted distribution of the kth element, is the estimated distribution of the reconstructed super-prior information, which is used to estimate the number of codewords required for the transmission of the reconstructed super-prior information, θ k ={φ k ,ψ k } is a set of learnable parameters.

11. The image compression method for machine and human vision according to claim 10, characterized in that: The parameters of the quantized convolutional neural network for N machine vision tasks are obtained by optimizing the bit rate and task accuracy: in, represents the loss function related to the accuracy of the i-th machine vision task, λ i >0 is the preset parameter used to balance the bit rate and task accuracy corresponding to the i-th machine vision task, R i is the bit rate of the i-th machine vision task, ω i >0 is the preset parameter used to balance between different machine vision tasks.

12. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it can be used to execute an image encoding method for machine and human vision as described in any one of claims 1-8, or an image decoding method for machine and human vision as described in claim 9, or an image compression method for machine and human vision as described in any one of claims 10-11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to execute an image encoding method for machine and human vision as described in any one of claims 1-8, or an image decoding method for machine and human vision as described in claim 9, or an image compression method for machine and human vision as described in any one of claims 10-11.

Images