Image encoding, decoding method and compression method for machine and human vision

Abstract

The application provides an image encoding, decoding and compression method for machine vision and human vision, and the main process is as follows: obtaining a first feature map and a second feature of an image to be encoded, and obtaining hyper-prior information of the first feature map; obtaining quantization steps of the image to be encoded under different machine vision tasks and quantizing to obtain corresponding feature maps; modeling the above feature maps as Gaussian distribution, and predicting corresponding mean and variance through an entropy model; encoding the feature maps layer by layer according to the distribution and probability of each layer of feature maps to obtain feature map binary code streams; and merging the hyper-prior information binary code stream and the feature map binary code stream to obtain a compressed image binary code rate. The application adaptively adjusts the quantization steps of the feature map quantization, forms a transmission feature code stream for human vision and different machine vision tasks, can more efficiently utilize the information in the feature map, reduces the transmission code rate while ensuring the performance of the machine vision task, and balances the rate distortion performance for human vision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image encoding and decoding method and compression method for machine and human vision. BACKGROUND

[0002] In recent years, end-to-end image compression coding technology based on deep learning has developed rapidly. However, many image coding transmission scenarios are not for human vision services, but for machines to perform computer vision tasks on decoded images. Therefore, the main goal of the end-to-end image compression model is not only to maintain visual quality, but also to maintain the task accuracy of machine tasks under a given bit rate. In 2019, L. Duan published "Video coding for machines: A paradigm of collaborative compression and intelligent analytics" in the IEEE Transactions on Image Processing journal, which proposed an end-to-end image compression coding scheme based on deep learning for machine vision. In 2020, Y. Hu published "Towards coding for human and machine vision: A scalable image coding approach" in the IEEE International Conference on Multimedia and Expo, which proposed a scalable coding method to realize an image compression coding framework for both human vision and machine vision, providing a wider application scenario for image compression coding. In 2022, H. Choi published "Scalable Image Coding for Humans and Machines" in the IEEE Transactions on Image Processing journal, which proposed an image coding scheme for machine vision combined with human vision based on fixed feature channel grouping. According to the complexity of the machine task, the number of transmitted feature groups is selected, which greatly improves the task accuracy of the machine task under a given bit rate with little impact on human vision performance.

[0003] However, since the feature channel grouping is fixed, the specified feature channel group is not the optimal feature channel group for different computer vision tasks, which leads to redundant code streams and suboptimal performance of machine tasks. SUMMARY

[0004] Aiming at the defects in the prior art, the present application aims to provide an image encoding and decoding method and compression method for machine and human vision.

[0005] In a first aspect, the present application provides an image encoding method for machine and human vision, comprising:

[0006] For an image to be compressed, a multi-channel feature map of the image is generated by an encoding convolutional neural network as a first feature map; the first feature map is quantized according to a quantization step of 1 to obtain a feature map for human vision coding as a second feature map;

[0007] For the first feature map, hyper-prior information is extracted by a hyper-prior encoding convolutional neural network, and a hyper-prior information binary code stream is obtained, and then the hyper-prior information binary code stream is decoded by a hyper-prior decoding convolutional neural network to generate reconstructed hyper-prior information;

[0008] The quantization step for machine vision coding is estimated using the reconstructed hyper-prior information, and is adjusted to be an odd number, and the first feature map is quantized into quantized feature maps for N machine vision tasks according to the quantization step, where N is a positive integer;

[0009] The distribution of the quantized feature maps for the Nth machine vision task to the first machine vision task and the distribution of the second feature map are predicted using the entropy model in sequence using the reconstructed hyper-prior information; probability estimation is performed according to the predicted distribution, and arithmetic coding is performed according to the estimated probability to obtain layer-by-layer feature Figure Two binary code streams, and the feature Figure Two binary code streams are obtained by merging.

[0010] The hyper-prior information binary code stream and the feature Figure Two binary code stream are merged to obtain a compressed image binary code stream.

[0011] Preferably, for the first feature map, hyper-prior information is extracted by a hyper-prior encoding convolutional neural network, and a hyper-prior information binary code stream is obtained, and then the hyper-prior information binary code stream is decoded by a hyper-prior decoding convolutional neural network to generate reconstructed hyper-prior information, comprising:

[0012] A hyper-prior feature map is obtained from the first feature map by a hyper-prior encoding convolutional neural network;

[0013] The hyper-prior feature map is quantized to obtain a quantized hyper-prior feature map;

[0014] The quantized hyper-prior feature map is subjected to feature-independent distribution estimation, and the quantized hyper-prior feature map is subjected to arithmetic coding using the distribution estimation to obtain a hyper-prior information binary code stream;

[0015] recovering hyper-prior information from the hyper-prior information binary stream by using arithmetic decoding;

[0016] obtaining reconstructed hyper-prior information from the recovered hyper-prior information by using a hyper-prior decoding convolutional neural network;

[0017] The hyper-prior decoding convolutional neural network is an inverse convolutional neural network corresponding to the hyper-prior encoding convolutional neural network.

[0018] Preferably, the reconstructed hyper-prior information is used to estimate a quantization step size for machine vision coding, and the quantization step size is adjusted to be an odd number, and the first feature map is quantized into quantized feature maps for N machine vision tasks according to the quantization step size, where N is a positive integer, including:

[0019] According to the information amount relationship required by the machine vision tasks, indexes i = 1,…,N of the N machine vision tasks are defined, and the smaller the index, the greater the information amount required by the machine vision task; the machine vision tasks include image classification, object detection, semantic segmentation, and instance segmentation;

[0020] The reconstructed hyper-prior information is used to obtain a quantization step size relationship between the N machine vision tasks;

[0021] According to the quantization step size corresponding relationship between different machine vision tasks, a quantization step size for machine vision coding of different tasks is obtained;

[0022] The first feature map is quantized using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to the N machine vision tasks.

[0023] Preferably, the reconstructed hyper-prior information is used to obtain a quantization step size relationship between the N machine vision tasks, including:

[0024] For i = 1,…,N, a quantization convolutional neural network is used to estimate a quantization step size relationship q i , between the first feature map and the i+1th machine vision task.

[0025] The reconstructed hyper-prior information is input into a convolutional layer Conv1 with a kernel size of 3×3 3×3 is transformed, and LeakyReLU is used as a nonlinear mapping layer to obtain latent layer quantization step size relationship information

[0026]

[0027] The LeakyReLU function is:

[0028]

[0029] wherein, a is a positive value parameter for regulating the gradient of negative value;

[0030] quantizing the latent layer step size relationship information is up-sampled, and sequentially passes through a deconvolutional layer DeConv1 using a SoftPlus nonlinear mapping layer 3×3 and a deconvolutional layer DeConv2 using a LeakyReLU nonlinear mapping layer 3×3 to obtain a coarse quantization step size relationship

[0031]

[0032] wherein, the expression of the SoftPlus function is:

[0033] SoftPlus(x) = log(1 + e x )

[0034] inputting the coarse quantization step size relationship into a convolutional layer Conv2 with a kernel size of 3x3 3×3 using a SoftPlus nonlinear mapping layer and adding a bias of 1 to ensure that the quantization step size relationship is not less than 1, to obtain a continuous quantization step size relationship

[0035]

[0036] adjusting the continuous quantization step size relationship so that the quantization step size is approximately an odd number, to obtain a quantization step size relationship q between the i-th machine vision task and the i+1-th machine vision task i , specifically:

[0037]

[0038] wherein, return the maximum positive integer not greater than q.

[0039] Preferably, the quantization step size corresponding to different machine vision tasks is obtained according to the quantization step size corresponding relationship of different machine vision tasks, including:

[0040] for i = 1, …, N, estimating the quantization step size Q i of the first feature map under the i-th machine vision task:

[0041]

[0042] Preferably, the first feature map is quantized by using the machine vision-oriented coding quantization step to obtain quantized feature maps corresponding to N machine vision tasks, specifically:

[0043] For i = 1, …, N, the quantized feature map corresponding to the i-th machine vision task is obtained is:

[0044]

[0045] Wherein y is the first feature map, and Round(y) returns the result of rounding the decimal part of y.

[0046] Preferably, the distribution of the quantized feature map of the N-th machine vision task to the first machine vision task is predicted in turn by using the entropy model with the reconstructed hyper-prior information, the distribution of the second feature map is predicted, the probability is estimated according to the predicted distribution, the arithmetic coding is performed according to the estimated probability, and the layer-by-layer feature Figure Two stream is obtained, and the feature Figure Two stream is obtained by merging, comprising:

[0047] The quantized feature map of the N-th machine vision task is parameterized as a Gaussian distribution, and the entropy model is used to predict the mean and variance of the Gaussian distribution by using the reconstructed hyper-prior information.

[0048] The quantized feature map of the N-th machine vision task is estimated according to the predicted mean and variance of the Gaussian distribution, and the arithmetic coding is performed according to the estimated probability to obtain the first layer feature Figure Two stream;

[0049] The Gaussian distribution mean and variance of the quantized feature map of the N-th machine vision task to the first machine vision task are predicted in turn by using the entropy model with the reconstructed hyper-prior information, the probability is estimated according to the predicted mean and variance of the Gaussian distribution, and the arithmetic coding is performed according to the estimated probability to obtain the second layer to the N-th layer feature Figure Two stream;

[0050] The Gaussian distribution mean and variance of the second feature map are predicted by using the entropy model with the reconstructed hyper-prior information, the probability is estimated according to the predicted mean and variance of the Gaussian distribution, and the arithmetic coding is performed according to the estimated probability to obtain the N+1-th layer feature Figure Two stream;

[0051] The N+1-th layer feature Figure Two stream is obtained by merging in the order of the first layer to the N+1-th layer, and the feature Figure Two stream is obtained. ​​

[0052] According to a second aspect of the present application, there is provided a method for decoding an image for machine and human vision, corresponding to any of the methods for encoding an image for machine and human vision, comprising:

[0053] decomposing the compressed image binary code stream to obtain a hyper-prior information binary code stream and a feature Figure Two binary code stream; decomposing the feature Figure Two binary code stream to obtain N+1 layers of features Figure Two binary code stream;

[0054] recovering hyper-prior information from the hyper-prior information binary code stream by arithmetic decoding; using an inverse convolutional neural network corresponding to the hyper-prior encoding convolutional neural network as a hyper-prior decoding convolutional neural network to obtain reconstructed hyper-prior information from the recovered hyper-prior information;

[0055] using a quantization convolutional neural network to estimate quantization steps corresponding to N machine vision tasks from the reconstructed hyper-prior information, and adjusting the quantization steps to be odd numbers; using an entropy model to predict the mean and variance of the Gaussian distribution of the quantized feature maps of the Nth to 1st machine vision tasks one by one using the reconstructed hyper-prior information combined with the quantization steps of each machine vision task layer; and decoding the features Figure Two binary code stream layer by layer from the 1st layer to the Nth layer according to the mean and variance of the Gaussian model to obtain quantized feature maps corresponding to the reconstructed Nth to 1st machine vision tasks; using an entropy model to predict the mean and variance of the Gaussian distribution of the second feature map using the reconstructed hyper-prior information; and decoding the N+1 layer features Figure Two binary code stream according to the mean and variance of the Gaussian model to obtain a reconstructed second feature map;

[0056] obtaining a reconstructed image for human vision by a decoding convolutional neural network corresponding to the encoding convolutional neural network using the reconstructed second feature map;

[0057] obtaining reconstructed images for machine vision tasks by a decoding convolutional neural network corresponding to the encoding convolutional neural network using the reconstructed feature maps of the N machine vision tasks.

[0058] According to a third aspect of the present application, there is provided a method for compressing an image for machine and human vision, comprising:

[0059] obtaining an image to be compressed;

[0060] compressing the image using any of the methods for encoding an image for machine and human vision to obtain a compressed image binary code stream;

[0061] The image decoding method for machine and human vision is used to realize decompression of the compressed image binary code stream, so as to obtain the reconstructed image for human vision and machine vision.

[0062] Preferably, the parameters of the encoding convolutional neural network, the decoding convolutional neural network, the hyper-prior encoding convolutional neural network, the hyper-prior decoding convolutional neural network and the entropy model are obtained by rate-distortion optimization:

[0063]

[0064] wherein the distortion x is the image to be compressed, is the reconstructed image, and λ>0 is a preset parameter for balancing the distortion and the code rate, and the code rate R is obtained according to the second feature map The Gaussian distribution of each element is calculated as follows:

[0065]

[0066] wherein, is the feature of the kth element, is the predicted distribution of the kth element, is the estimated distribution of the reconstructed hyper-prior information, and is used to estimate the number of code words required for transmission of the reconstructed hyper-prior information, and θ k ={φ k ,ψ k} is a set of learnable parameters.

[0067] Preferably, the parameters of the quantized convolutional neural network of the N machine vision tasks are obtained by optimization of the code rate and the task accuracy:

[0068]

[0069] wherein, represents a loss function related to the accuracy of the ith machine vision task, and λ i >0 is a preset parameter for balancing the code rate and the task accuracy of the ith machine vision task, and R i is the code rate of the ith machine vision task, and ω i >0 is a preset parameter for balancing different machine vision tasks.

[0070] According to a fourth aspect of the present application, a terminal comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor is configured to execute the program to implement any one of the image encoding methods for machine and human vision, or the image decoding methods for machine and human vision, or the image compression methods for machine and human vision.

[0071] According to a fifth aspect of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, is adapted to perform any of the machine and human vision oriented image encoding method, or the machine and human vision oriented image decoding method, or any of the machine and human vision oriented image compression method.

[0072] Compared with the prior art, the embodiments of the present application have at least one of the following beneficial effects:

[0073] The machine and human vision oriented image encoding, decoding and compression method in the embodiments of the present application adaptively adjusts the quantization step of the feature map quantization, forms a transmission feature code stream for human vision and different machine vision tasks, can more efficiently utilize the information in the feature map, reduces the transmission code rate while ensuring the performance of the machine vision task, and balances the rate distortion performance for human vision.

[0074] The machine and human vision oriented image encoding, decoding and compression method in the embodiments of the present application can enable a single code stream to perform multiple different machine vision tasks and obtain a performance close to that of a single machine vision task.

[0075] The machine and human vision oriented image encoding, decoding and compression method in the embodiments of the present application can generate a single binary code stream that can be truncated through the quantization step, form a reconstructed image for different tasks of human vision and machine vision, adapt to terminals performing different tasks, and does not need to switch different code stream versions for multiple different tasks, thereby improving user experience and reducing the overall bandwidth demand of network transmission.

[0076] The machine and human vision oriented image encoding, decoding and compression method in the embodiments of the present application can more efficiently utilize the feature map to perform downstream tasks of the image, control the quantization step of the feature map, and maximize the reduction of the transmission code rate on the basis of meeting the task performance index. BRIEF DESCRIPTION OF DRAWINGS

[0077] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments thereof, read in conjunction with the accompanying drawings:

[0078] Figure One A flowchart of the machine and human vision oriented image encoding method in an embodiment of the present application;

[0079] Figure Two A flowchart of the machine and human vision oriented image encoding method in a preferred embodiment of the present application;

[0080] Figure ThreeA flow chart of the image decoding method for machine and human vision in a preferred embodiment of the present application;

[0081] Figure Four A flow chart of the image compression method for machine and human vision in a preferred embodiment of the present application;

[0082] Figure Five A flow chart of the training method of the image compression model for machine and human vision in a preferred embodiment of the present application. DETAILED DESCRIPTION

[0083] The present application will be described in detail below with specific embodiments and drawings. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made. These are within the scope of the present application.

[0084] Referring to Figure One The flow chart of the image encoding for machine and human vision in an embodiment of the present application is shown. The present embodiment provides an image encoding method for machine vision, comprising steps S100-S500.

[0085] Specifically:

[0086] S100. For the image to be compressed, a multi-channel feature map of the image is generated using an encoding convolutional neural network as a first feature map; the first feature map is quantized according to a quantization step size of 1 to obtain a feature map for human vision encoding as a second feature map;

[0087] S200. For the first feature map obtained in S100, super-prior information is extracted by a super-prior encoding convolutional neural network, and a super-prior information binary code stream is obtained, and the reconstructed super-prior information is generated according to the super-prior information binary code stream;

[0088] S300. The reconstructed super-prior information obtained in S200 is used to estimate the quantization step size for machine vision encoding, and is adjusted to be an odd number to meet the embedded quantization condition, eliminate codeword redundancy, improve codeword utilization, and quantize the first feature map obtained in S100 to quantized feature maps for N machine vision tasks according to the quantization step size, where N is a positive integer;

[0089] S400. The reconstructed super-prior information obtained in S200 is used to sequentially predict the distribution of the quantized feature maps for the Nth machine vision task to the first machine vision task and the second feature map using an entropy model; probability estimation is performed according to the predicted distribution, and arithmetic coding is performed according to the estimated probability to obtain a layer-by-layer feature Figure Two binary code stream, and the featureFigure Two binary code stream;

[0090] S500. Merge the super-prior information binary code stream obtained in S200 and the feature Figure Two binary code stream, to obtain a compressed image binary code stream.

[0091] In step S100 of the above embodiment, the first feature map mentioned is a three-dimensional feature map of the image to be compressed, and the dimensions of the three-dimensional feature include feature channels, feature height, and feature width. In S400, it is mentioned that the entropy model network is implemented using existing technologies such as super-prior network and spatial context model, such as using a super-prior decoding convolutional neural network to recover a super-prior feature map from the super-prior information, corresponding to the mean, variance, etc. of the Gaussian distribution. When encoding the feature map corresponding to each machine vision task and the second feature map, an incremental encoding scheme is used to achieve a layered code stream, and the incremental encoding scheme is implemented using existing technologies.

[0092] The above embodiments of the present application can efficiently extract more needed features and information for machine vision, while reducing the transmission of redundant features for machine vision tasks, and can reduce the image compression code rate overhead while effectively improving the accuracy of downstream tasks. At the same time, using the idea of progressive encoding, the code stream obtained by encoding can be layered, and a single code stream can simultaneously perform multiple machine vision tasks, with higher efficiency and reduced code stream transmission overhead under multiple tasks.

[0093] Figure Two A flowchart of the image encoding method for machine and human vision in a preferred embodiment of the present application. In this preferred embodiment, the image encoding for machine vision can include more detailed steps S101-S109, specifically:

[0094] S101, three-dimensional feature extraction: the feature map obtained by the image to be compressed through the encoding convolutional neural network is obtained, and the three-dimensional feature, i.e. the first feature map, is obtained;

[0095] S102, super-prior information extraction and quantization: a super-prior encoding convolutional neural network is used to obtain super-prior information from the first feature map; the super-prior information is rounded and quantized to an integer;

[0096] S103, super-prior information encoding: a distribution estimation independent of each feature is established for the super-prior information, and arithmetic encoding is used to encode the super-prior information using the distribution estimation, to obtain a super-prior information binary code stream;

[0097] S104, super-prior information decoding: arithmetic decoding is performed on the super-prior information binary code stream to obtain recovered super-prior information; a super-prior decoding convolutional neural network is used to obtain a super-prior feature map from the recovered super-prior information;

[0098] S105, quantization: quantizing the first feature map into an integer to obtain a second feature map; Figure Two S105, quantization: quantizing the first feature map into an integer to obtain a second feature map;

[0099] S106, machine vision task quantization: obtaining the quantization step relationship between different task layers according to the recovered hyper-prior information and through the quantization convolutional neural network; then, obtaining the quantization step under different tasks according to the obtained quantization step correspondence relationship between different task layers; finally, quantizing the first feature map using the quantization step for machine vision coding to obtain the feature map corresponding to different machine vision tasks.

[0100] S107, probability entropy modeling: parameterizing the feature distribution of the feature map as a Gaussian distribution, and predicting the mean and variance of the corresponding Gaussian distribution using hyper-prior information and context model;

[0101] S108, three-dimensional feature progressive coding: performing probability estimation on all feature maps according to the predicted mean and variance of all feature map Gaussian distributions, and using existing progressive coding technology to encode all feature maps layer by layer to obtain the feature binary code stream of three-dimensional features.

[0102] S109, further combining the feature binary code stream with the hyper-prior feature binary code stream to obtain the compressed image binary code stream.

[0103] In order to ensure the optimal execution effect of each step, some embodiments of the present application provide an optimal scheme for S200, S300 and S400, respectively.

[0104] In an embodiment of the present application, an optimal scheme for implementing step S200 is provided, and the specific steps are as follows:

[0105] S21, obtaining a hyper-prior feature map from the first feature map through a hyper-prior encoding convolutional neural network;

[0106] S22, quantizing the hyper-prior feature map obtained in S21 to obtain a quantized hyper-prior feature map;

[0107] S23, estimating the distribution of the quantized hyper-prior feature map obtained in S22, and using the distribution estimation to perform arithmetic coding on the quantized hyper-prior feature map to obtain a hyper-prior information binary code stream;

[0108] S24, using arithmetic decoding to recover the hyper-prior information from the hyper-prior information binary code stream obtained in S23;

[0109] S25, obtaining reconstructed hyper-prior information from the hyper-prior information recovered in S24 through a hyper-prior decoding convolutional neural network;

[0110] The super-prior decoding convolutional neural network is an inverse convolutional neural network corresponding to the super-prior encoding convolutional neural network.

[0111] In the above embodiment, the steps S21-S25 realize the encoding of the prior information of the first and second feature maps to be encoded, reduce the encoding overhead of the first and second feature maps, and improve the overall encoding efficiency.

[0112] In an embodiment of the present application, a preferred method for implementing step S300 is provided, and the specific steps are as follows:

[0113] S31, according to the information amount relationship required by the machine vision task, the indexes of the N machine vision tasks are defined as i=1,…,N, and the smaller the index, the greater the information amount required by the machine vision task; the machine vision task includes image classification, object detection, semantic segmentation and instance segmentation;

[0114] S32, the quantization step length relationship between the N machine vision tasks is obtained by using the reconstructed super-prior information determined in S200;

[0115] S33, according to the quantization step length corresponding relationship between different machine vision tasks, the quantization step length for machine vision coding of different tasks is obtained;

[0116] S34, the first feature map is quantized by using the quantization step length for machine vision coding obtained in S33, and the quantized feature map corresponding to the N machine vision tasks is obtained.

[0117] In the above embodiment, the multi-feature map quantization realized by steps S31-S34 can realize the transmission of the contents required by multiple vision tasks by a single code stream, and improves the application range of the above method.

[0118] In a preferred embodiment, S32 is implemented, and the quantization step length relationship between the N machine vision tasks is obtained by using the reconstructed super-prior information, including:

[0119] For i=1,…,N, the quantization convolutional neural network is used to estimate the quantization step length relationship q i of the first feature map between the i-th machine vision task and the i+1-th machine vision task.

[0120] The reconstructed super-prior information is input into a convolutional layer Conv1 with a kernel size of 3×3 3×3 is transformed, and LeakyReLU is used as a nonlinear mapping layer to obtain the latent layer quantization step length relationship information

[0121]

[0122] wherein the LeakyReLU function is:

[0123]

[0124] wherein a is a positive value parameter for regulating the gradient of negative values;

[0125] Quantization step size relationship information of the latent layer Up-sampling, sequentially through the de-convolutional layer DeConv1 using the SoftPlus non-linear mapping layer 3×3 And the de-convolutional layer DeConv2 using the LeakyReLU non-linear mapping layer 3×3 , to obtain the rough quantization step size relationship

[0126]

[0127] wherein the SoftPlus function expression is as follows:

[0128] SoftPlus(x)=log(1+e x )

[0129] The rough quantization step size relationship is input into the convolutional layer Conv2 with a kernel size of 3x3 3×3 , using the SoftPlus non-linear mapping layer and adding a bias of 1 to ensure that the quantization step size relationship is not less than 1, to obtain the continuous quantization step size relationship

[0130]

[0131] Adjust the continuous quantization step size relationship to make the quantization step size approximately odd, meet the embedded quantization condition, eliminate codeword redundancy, and improve codeword utilization, to obtain the quantization step size relationship q between the i-th machine vision task and the i+1-th machine vision task i , the formula is:

[0132]

[0133] wherein, Return the maximum positive integer not greater than q.

[0134] In a preferred embodiment, S33 is implemented to obtain the machine vision-oriented coding quantization step size of different tasks according to the quantization step size corresponding relationship between different machine vision tasks, including:

[0135] For i=1,…,N, estimate the quantization step size Q of the first feature map under the i-th machine vision task: i ​

[0136]

[0137] In a preferred embodiment, S34 is implemented to quantize the first feature map using a machine vision-oriented quantization step to obtain N quantized feature maps corresponding to the machine vision tasks, specifically:

[0138] For i = 1, …, N, obtain the quantized feature map corresponding to the i-th machine vision task is:

[0139]

[0140] where y is the first feature map, and Round(y) returns the result of rounding the decimal part of y.

[0141] In the above embodiments of the present application, by adjusting the quantization step of the feature map under the machine vision task, the present application can efficiently retain the more needed information of machine vision, reduce the information redundant to machine vision, and reduce the image compression code rate overhead while effectively improving the accuracy of the downstream task.

[0142] In a preferred embodiment of the present application, S400 is implemented, and the specific process is as follows:

[0143] S41, quantized feature map of the N-th machine vision task The parameterized modeling is a Gaussian distribution, and the entropy model is used to predict the mean and variance of the Gaussian distribution using the reconstructed hyper-prior information;

[0144] According to the predicted mean and variance of the Gaussian distribution, the quantized feature map of the N-th machine vision task is estimated, and according to the estimated probability, arithmetic coding is performed to obtain the first layer feature Figure Three in the binary code stream;

[0145] S42, the entropy model is used to predict the Gaussian distribution mean and variance of the quantized feature map of the N-1-th to the 1st machine vision task in turn using the reconstructed hyper-prior information, the probability is estimated according to the predicted mean and variance of the Gaussian distribution, and the arithmetic coding is performed according to the estimated probability to obtain the second layer to the N-th layer feature Figure Two in the binary code stream;

[0146] S43, the entropy model is used to predict the Gaussian distribution mean and variance of the second feature map using the reconstructed hyper-prior information, the probability is estimated according to the predicted mean and variance of the Gaussian distribution, and the arithmetic coding is performed according to the estimated probability to obtain the N+1-th layer feature Figure Two in the binary code stream;

[0147] S44, according to the order of the first layer to the N+1-th layer, the N+1-th layer feature is merged.Figure Two obtaining features Figure Two obtaining features

[0148] The above embodiment of the present application uses the idea of progressive coding, and the obtained code stream can be layered, a single code stream can simultaneously perform multiple machine vision tasks, has higher efficiency, and reduces the code stream transmission overhead under multiple tasks.

[0149] Based on the same inventive concept, another embodiment of the present application provides an image decoding method for machine and human vision, a flowchart is shown in Figure Two , and includes:

[0150] S201, code stream decomposition: decomposing the compressed image binary code stream to obtain features Figure Two binary code stream and hyper-prior information binary code stream; decomposing the features Figure Two binary code stream to obtain N+1 layers of features Figure Two binary code stream

[0151] S202, hyper-prior information decoding: performing arithmetic decoding on the hyper-prior information binary code stream to obtain recovered hyper-prior information; using a hyper-prior decoding convolutional neural network on the recovered hyper-prior information to obtain reconstructed hyper-prior information

[0152] S203, probability entropy modeling: the structure and parameter values of this module are completely the same as those in the embodiment shown in Figure Two ; using the entropy model to predict the mean and variance of the Gaussian distribution of the quantized feature map of the Nth to first machine vision task one by one using the reconstructed hyper-prior information, and decoding the features from the first layer to the Nth layer according to the mean and variance of the Gaussian model Figure Four binary code stream to obtain the quantized feature map corresponding to the Nth to first machine vision task; using the entropy model to predict the mean and variance of the Gaussian distribution of the second feature map using the reconstructed hyper-prior information, and decoding the N+1 layer features Figure One binary code stream to obtain the second feature map

[0153] S204, machine vision task quantization step prediction: the structure of this module is completely the same as the part of calculating the quantization step in S7, and the quantization step of the feature map corresponding to each machine vision task is predicted from the recovered reconstructed hyper-prior information. Specifically: using a quantization convolutional neural network to estimate the quantization step corresponding to the N machine vision tasks from the reconstructed hyper-prior information, and adjusting it to be an odd number.

[0154] S205, three-dimensional feature progressive decoding: according to the mean and variance of the Gaussian model of S203 and the quantization step of the feature map corresponding to the machine vision task of S204, the features Figure ThreeThe binary code stream is progressively decoded, and the progressive decoding uses existing technology to obtain the recovered feature map corresponding to the machine vision task and the second feature map corresponding to human vision layer by layer.

[0155] S206, image reconstruction: the second feature map reconstructed in S205 is input into a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain a reconstructed image for human vision.

[0156] The N machine vision task reconstruction feature maps obtained in S205 are input into a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain reconstructed images for machine vision tasks.

[0157] The image decoding method for machine vision and human vision based on adaptive quantization step size in the embodiment is used for corresponding Figure Five The image encoding method for machine vision shown in the embodiment. The parts not described in detail above can be referred to the corresponding description in the embodiment shown in Figure Two The parts not described in detail above can be referred to the corresponding description in the embodiment shown in

[0158] According to the same inventive concept, in other embodiments of the present application, an image compression method for machine and human vision is provided, and the main steps are as follows:

[0159] First, obtain the image to be compressed;

[0160] Second, use the image encoding method for machine and human vision in the above embodiment to compress the image to obtain a compressed image binary code stream;

[0161] Third, use the image decoding method for machine and human vision in the above embodiment to decompress the compressed image binary code stream to obtain a reconstructed image for human vision and machine vision.

[0162] Further, in a specific embodiment, an image compression method for machine and human vision is used to compress all images in the COCO2017 test set, and a flowchart of the method is shown in Figure One , which includes:

[0163] S301, use the extraction module: obtain the image to be compressed in sequence;

[0164] S302, use the encoder: encode the image to be compressed according to the image encoding method for machine vision in the above embodiment to obtain a compressed image binary code stream.

[0165] Specifically, the channel number of the extracted three-dimensional feature is 192, and the down-sampling multiple is 16; the number of machine vision tasks is set to 2, which are target detection task and semantic segmentation task.

[0166] S303, using a decoder: decompressing the compressed image binary code stream according to the machine vision-oriented image decoding method to obtain a partially reconstructed image for two types of downstream image tasks and a reconstructed image for human vision.

[0167] The compression method in the above embodiments of the present application is implemented based on the machine and human vision-oriented image encoding method and the machine and human vision-oriented image decoding method shown in Figure One 、 2 The above parts not described in detail can be referred to the corresponding description in the above embodiments, and will not be described here again. Figure One

[0168] The compression method in the above embodiments of the present application can efficiently extract information more needed by machine vision, while reducing information redundant to machine vision, can reduce image compression code rate overhead, and effectively improve the accuracy of downstream tasks. At the same time, the code stream obtained by progressive encoding can be layered, and a single code stream can simultaneously perform multiple machine vision tasks, with higher efficiency and reduced code stream transmission overhead under multiple tasks.

[0169] For the above image compression method, the compression effect can be improved by training the model. For example, the training set is composed of all images of COCO2017-train data, and the model after training can be used for machine vision-oriented image encoding and decoding. Specifically, as shown in Figure One The image compression method includes:

[0170] S401, three-dimensional feature extraction: randomly cropping the images in the training set into images with a resolution size of 256x256, and inputting the images into an encoding convolutional neural network to obtain three-dimensional features. The encoding convolutional neural network is composed of three residual blocks, an attention mechanism module, three residual blocks, a convolutional layer, and an attention mechanism module, and the decoding neural network can also be other network structures, such as four-layer convolution cascading. The three-dimensional feature channel number is 128 or 192, and the height and width are both 16. Of course, the image resolution size and three-dimensional feature parameters in the above embodiments can be adjusted according to actual conditions, and are not limited to the description in this step, and the parameters in the following steps are also the same.

[0171] ​S402, super-prior information extraction and noise addition: input the three-dimensional features into a super-prior encoding convolutional neural network to obtain super-prior features, and to avoid the gradient problem caused by quantization, add noise uniformly distributed between -0.5 and 0.5 to the super-prior features to simulate actual quantization, wherein: the super-prior encoding convolutional neural network is composed of three convolutional layers; of course, the super-prior encoding neural network can also be other network structures, such as three residual modules in cascade, and is not limited to the second convolutional neural network in this step.

[0172] S403, super-prior code rate estimation: a distribution estimation independent of each feature is established for the super-prior information, and a super-prior code rate estimation is obtained using the distribution estimation;

[0173] S404, super-prior information acquisition: input the super-prior information with added noise into a super-prior decoding convolutional neural network to obtain super-prior information, wherein: the super-prior decoding convolutional neural network is composed of three convolutional layers in cascade;

[0174] S405, machine vision quantization step prediction: input the super-prior information with added noise into a quantization convolutional neural network to obtain the feature map quantization step relationship under each machine vision task, and further obtain the feature map quantization step under each machine vision task, and by quantizing the result of the quantization convolutional neural network to an odd number, the quantization steps of each layer satisfy the relationship of odd multiples, and here, to avoid the gradient problem caused by quantization, the gradient is set to 1 when quantization is performed;

[0175] S406, feature noise addition: to avoid the gradient problem caused by quantization, according to the corresponding quantization step Q of the feature map, add noise uniformly distributed between -0.5Q and 0.5Q to the three-dimensional features to simulate actual quantization; if it is human vision, then Q = 1;

[0176] S407, Gaussian distribution probability modeling: the structure of this module is exactly the same as that of the probability entropy modeling in the embodiment shown in Figure One

[0177] S408, feature code rate estimation: obtain the feature code rate estimation according to the predicted mean and variance of the Gaussian distribution;

[0178] S409, total code rate estimation: add the super-prior feature code rate estimation and the feature code rate estimation to obtain the total code rate estimation;

[0179] ​S410, image reconstruction: input the three-dimensional features after adding noise into the decoding convolutional neural network to obtain a reconstructed image. The decoding convolutional neural network is sequentially cascaded by an attention mechanism module, three residual blocks, an attention mechanism module, three residual blocks, and an up-sampling convolutional layer. The decoding convolutional neural network can also be other network structures, which corresponds to the encoding convolutional neural network. The above image compression method for machine vision also needs to meet the image compression model for human vision. The random gradient descent method can be further used for stage optimization. In the first stage, the machine vision quantization step prediction module is disabled, the quantization step is set to Q=1, and only the encoding convolutional neural network, the decoding convolutional neural network, the hyper-prior encoding convolutional neural network, the hyper-prior decoding convolutional neural network, and the entropy model are optimized. The loss function of the first stage is:

[0180]

[0181] wherein the distortion x is the image to be compressed, is the reconstructed image, and λ>0 is a preset parameter for balancing the distortion and the code rate. The code rate R is calculated according to the second feature map The Gaussian distribution of each element is obtained:

[0182]

[0183] wherein is the feature of the kth element, is the predicted distribution of the kth element, and θ k ={φ k ,ψ k} is a set of learnable parameters.

[0184] In the second stage, the encoding convolutional neural network, the decoding convolutional neural network, the hyper-prior encoding convolutional neural network, the hyper-prior decoding convolutional neural network, and the entropy model are fixed, and the machine vision quantization step prediction module is optimized. The parameters are optimized by the following formula:

[0185]

[0186] wherein represents the loss function related to the accuracy of the i th machine vision task, and λ i >0 is a preset parameter for balancing the code rate and the task accuracy of the i th machine vision task. The code rate R i of the i th machine vision task is calculated in the same way as the first stage code rate, and ω i >0 is a preset parameter for balancing different machine vision tasks.

[0187] Implementation effect:

[0188] In the above embodiment, the image compression model for machine vision is trained using the ImageNet dataset, and in the first stage, λ is set to 0.0018, 0.0035, 0.0067, and 0.0130 respectively for four different image compression models with different code rates, and the Adam optimizer is used for 2000000-step optimization, with the learning rate set to 10 -4 in the first 1800000 steps, then reduced to 10 -5 , and finally reduced to 10 -6 at 1950000 steps. In the second stage, the COCO2017-train dataset is used for training, and λ is set to 0.0018, 0.0035, 0.0067, and 0.0130 respectively for four different code rates, and the Adam optimizer is used for 200000-step optimization, with the learning rate set to 10 -3 in the first 50000 steps, then reduced to 10 -4 .

[0189] In the above embodiment, the image encoder of the four trained image compression models for machine vision is used to compress all images in the COCO2017-validation test set. The average code rate of the compressed images is 0.0884bpp, 0.1265bpp, 0.177bpp, and 0.2488bpp respectively; in the above embodiment, the image encoder of the four trained image compression models for machine vision is used to compress all images in the COCO2017-validation test set to obtain reconstructed images for machine vision tasks, and the machine vision tasks selected in the embodiment are target detection tasks based on Faster-RCNN and semantic segmentation tasks based on Mask-RCNN, the average code rate of the detection task is 0.063bpp, 0.093bpp, 0.142bpp, and 0.209bpp respectively, and the average intersection over union of the detection is 28.887%, 32.225%, 35.456%, and 36.802% respectively; the average code rate of the segmentation task is 0.07bpp, 0.103bpp, 0.148bpp, and 0.224bpp respectively, and the average intersection over union of the segmentation is 27.1%, 30.436%, 32.808%, and 34.147% respectively

[0190] Compared with the method using the benchmark neural network, the image compression method for machine vision provided in the above embodiment of the present application has better rate-distortion performance. Specifically, under the premise of the same code rate overhead, the accuracy of the reconstructed image for target detection task can be improved by 17.71%, and the accuracy of the reconstructed image for semantic segmentation task can be improved by 13.99%, and the rate-distortion performance is improved.

[0191] Based on the same inventive concept, in other embodiments of the present application, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, is configured to perform the machine and human visual oriented image encoding method, or the machine and human visual oriented image decoding method, or the machine and human visual oriented image compression method.

[0192] Optionally, the memory is configured to store the program; the memory can comprise volatile memory (e.g., random-access memory (RAM), such as static random-access memory (SRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), etc.), or non-volatile memory (e.g., flash memory). The memory is configured to store computer programs (e.g., application programs, functional modules, etc. for implementing the above method), computer instructions, etc., and the above computer programs and computer instructions can be stored in one or more memories.

[0193] The processor is configured to execute the computer program stored in the memory to implement each step of the method described in the above embodiments. For details, refer to the related description in the method embodiments.

[0194] The processor and the memory can be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor can be coupled and connected through a bus.

[0195] Based on the same inventive concept, in other embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, wherein the program, when executed by a processor, is configured to perform the machine and human visual oriented image encoding method, or the machine and human visual oriented image decoding method, or the machine and human visual oriented image compression method.

[0196] Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired computer program code means in the form of computer readable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a combination thereof. Also, functional

[0197] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code means.

[0198] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to the present application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure One one or more functions specified in one or more of the flowchart illustrations and / or block diagrams. ​ means for (an apparatus with) one or more functions specified in one or more of the flowchart illustrations and / or block diagrams.

[0199] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. ​ one or more functions specified in one or more of the flowchart illustrations and / or block diagrams. ​ means for (an apparatus with) one or more functions specified in one or more of the flowchart illustrations and / or block diagrams.

[0200] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. ​one or more processes and / or blocks ​ the steps of a function specified in one or more blocks.

[0201] The specific embodiments of the present application have been described. It is to be understood that the application is not limited to the specific embodiments described and that various modifications made to the embodiments described by a person skilled in the art without departing from the spirit of the application are within the scope of the application. The preferred features described above can be used in any combination without conflict.

Claims

1. An image coding method for machine and human vision, characterized in that, include: For the image to be compressed, a multi-channel feature map of the image is generated using an encoding convolutional neural network, which serves as the first feature map. The first feature map is quantized with a quantization step size of 1 to obtain a feature map oriented towards human visual encoding, which is used as the second feature map. For the first feature map, the super-prior information is extracted by the super-prior coding convolutional neural network and the super-prior information binary code stream is obtained. Then, the super-prior information binary code stream is decoded by the super-prior decoding convolutional neural network to generate reconstructed super-prior information. The quantization step size for machine vision coding is estimated using the reconstructed prior information and adjusted to an odd number. The first feature map is then quantized into quantized feature maps for N machine vision tasks based on the quantization step size, where N is a positive integer. Using the reconstructed prior information, the distribution of the quantized feature map and the distribution of the second feature map for the Nth to the 1st machine vision task are predicted sequentially using an entropy model; probability estimation is performed based on the predicted distribution, and arithmetic encoding is performed based on the estimated probability to obtain a layer-by-layer feature map binary code stream, which is then merged to obtain the feature map binary code stream. The binary code stream of the prior information and the binary code stream of the feature map are merged to obtain the compressed image binary code stream; The method involves estimating the quantization step size for machine vision coding using the reconstructed prior information and adjusting it to an odd number. Based on this quantization step size, the first feature map is quantized into quantized feature maps for N machine vision tasks, where N is a positive integer, including: Based on the information requirements of machine vision tasks, the indices i = 1, ..., N for N machine vision tasks are defined. The smaller the index, the greater the information required by the machine vision task. The machine vision tasks include image classification, object detection, semantic segmentation, and instance segmentation. The quantization step size relationship between N machine vision tasks is obtained by using the reconstructed prior information; Based on the correspondence between quantization step sizes of different machine vision tasks, the quantization step sizes for machine vision-oriented coding of different tasks are obtained. The first feature map is quantized using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks. The process of obtaining the quantization step size relationship between N machine vision tasks using the reconstructed prior information includes: For i = 1, ..., N, a quantized convolutional neural network is used to estimate the quantization step size relationship q of the first feature map between the i-th machine vision task and the (i+1)-th machine vision task. i Specifically: The reconstructed prior information Conv1, a convolutional layer with a kernel size of 3×3, is input. 3×3 A transformation is performed, using LeakyReLU as the nonlinear mapping layer, to obtain the latent quantization step size relationship information. The LeakyReLU function is as follows: Where α is a positive parameter used to control the gradient of negative values; The relationship information of the quantization step size of the subsurface Upsampling is performed by sequentially using the DeConv1 deconvolution layer of the SoftPlus nonlinear mapping layer. 3×3 and the deconvolutional layer DeConv2 using LeakyReLU nonlinear mapping layers 3×3 The rough quantization step size relationship is obtained. The SoftPlus function expression is as follows: SoftPlus(x)=log(1+e x ) The coarse quantization step size relationship Conv2 convolutional layer with a kernel size of 3×3 as input 3×3 Using a SoftPlus nonlinear mapping layer and adding a bias of 1, the quantization step size relationship is ensured to be no less than 1, thus obtaining a continuous quantization step size relationship. Adjust the relationship of continuous quantization step size By approximating an odd number for the quantization step size, we obtain the quantization step size relationship q between the i-th machine vision task and the (i+1)-th machine vision task. i Specifically: in, Return the largest positive integer not greater than q.

2. The image coding method for machine and human vision according to claim 1, characterized in that, For the first feature map, the process involves extracting prior information using a super-prior coding convolutional neural network and obtaining a binary code stream of the super-prior information. Subsequently, a super-prior decoding convolutional neural network is used to decode the binary code stream of the super-prior information to generate reconstructed super-prior information, including: The super-prior feature map is obtained from the first feature map by a super-prior coding convolutional neural network; The super-prior feature map is quantized to obtain a quantized super-prior feature map; The quantized super-prior feature map is subjected to a distribution estimation of mutually independent features, and the distribution estimation is used to perform arithmetic encoding on the quantized super-prior feature map to obtain a binary code stream of super-prior information. Arithmetic decoding is used to recover the prior information from the prior information binary code stream; Reconstructed hyperprior information is obtained from the recovered hyperprior information through a hyperprior decoding convolutional neural network; The super-prior decoding convolutional neural network is a deconvolutional neural network corresponding to the super-prior encoding convolutional neural network.

3. The image coding method for machine and human vision according to claim 2, characterized in that, The step of obtaining the quantization step size for machine vision encoding for different tasks based on the correspondence between quantization step sizes for different machine vision tasks includes: For i = 1, ..., N, estimate the quantization step size Q of the first feature map under the i-th machine vision task. i :

4. The image coding method for machine and human vision according to claim 3, characterized in that, The process of quantizing the first feature map using the quantization step size for machine vision coding to obtain quantized feature maps corresponding to N machine vision tasks is as follows: For i = 1, ..., N, obtain the quantized feature map corresponding to the i-th machine vision task. for: Where y is the first feature map, and Round(y) returns the result after rounding the decimal part of y.

5. The image coding method for machine and human vision according to claim 1, characterized in that, The process involves using reconstructed prior information to sequentially predict the distribution of the quantized feature maps and the distribution of the second feature map for the Nth to the 1st machine vision task using an entropy model, performing probability estimation based on the predicted distribution, performing arithmetic encoding based on the estimated probability to obtain a layer-by-layer feature map binary code stream, and merging them to obtain the feature map binary code stream, including: Quantize the feature map of the Nth machine vision task The parameterized model is a Gaussian distribution, and the entropy model is used to predict the mean and variance of the Gaussian distribution using the reconstructed prior information; The quantized feature map of the Nth machine vision task is generated based on the predicted mean and variance of the Gaussian distribution. Probability estimation is performed, and arithmetic encoding is performed based on the estimated probability to obtain the binary code stream of the first layer feature map; The mean and variance of the Gaussian distribution of the quantized feature maps of the N-1th to the 1st machine vision task are predicted sequentially using the entropy model with the reconstructed prior information. Probability estimation is performed based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is performed based on the estimated probability to obtain the binary code stream of the feature maps of the 2nd to Nth layers. Using the reconstructed prior information, the mean and variance of the Gaussian distribution of the second feature map are predicted by the entropy model. Probability estimation is performed based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is performed based on the estimated probability to obtain the binary code stream of the N+1 layer feature map. Following the order of layers 1 to N+1, the binary code streams of the feature maps in layer N+1 are merged to obtain the binary code stream of the feature maps.

6. An image decoding method for machine and human vision, used to correspondingly decode the image encoding method for machine and human vision according to any one of claims 1-5, characterized in that, include: Decompose the compressed image binary code stream to obtain the prior information binary code stream and the feature map binary code stream; Decompose the binary code stream of the feature map to obtain an N+1 layer binary code stream of the feature map; Arithmetic decoding is used to recover the prior information from the binary code stream of the prior information; the deconvolutional neural network of the corresponding prior coding convolutional neural network is used as the prior decoding convolutional neural network to obtain the reconstructed prior information from the recovered prior information. A quantized convolutional neural network is used to estimate the quantization step size corresponding to N machine vision tasks from the reconstructed prior information and adjust it to an odd number. An entropy model is used to use the reconstructed prior information and combine it with the quantization step size of each machine vision task layer to predict the mean and variance of the Gaussian distribution of the quantized feature map of the Nth to the 1st machine vision task. Based on the mean and variance of the Gaussian distribution, the binary code stream of the feature map is decoded layer by layer from the 1st to the Nth layer to obtain the quantized feature map of the corresponding reconstructed Nth to the 1st machine vision task. The mean and variance of the Gaussian distribution of the second feature map are predicted using the entropy model with the reconstructed prior information. The binary code stream of the N+1 layer feature map is then decoded based on the mean and variance of the Gaussian distribution to obtain the reconstructed second feature map. The reconstructed second feature map is then passed through a decoding convolutional neural network corresponding to the encoding convolutional neural network to obtain a reconstructed image for human vision. The reconstructed feature maps of N machine vision tasks are passed through the corresponding encoding convolutional neural network and the decoding convolutional neural network to obtain the reconstructed image for the machine vision task.

7. An image compression method for machine and human vision, characterized in that, include: Obtain the image to be compressed; The image to be compressed is compressed using the image coding method for machine and human vision as described in any one of claims 1-5 to obtain a compressed image binary bitstream; The image decoding method for machine and human vision described in claim 6 is used to decompress the compressed image binary stream to obtain a reconstructed image for human and machine vision.

8. The image compression method for machine and human vision according to claim 7, characterized in that, The parameters of the encoding convolutional neural network, decoding convolutional neural network, super-prior encoding convolutional neural network, super-prior decoding convolutional neural network, and entropy model are obtained through rate-distortion optimization. Among them, distortion x is the image to be compressed. This is a reconstructed image; λ>0 is a preset parameter used to balance distortion and bitrate; the bitrate R is based on the second feature map. The Gaussian distribution of each element is calculated as follows: in, It is the characteristic of the k-th element. It is the predicted distribution of the k-th element. To reconstruct the estimated distribution of the prior information, θ is used to estimate the number of codewords required for transmitting the reconstructed prior information. k ={φ k ,ψ k } is a set of learnable parameters.

9. The image compression method for machine and human vision according to claim 8, characterized in that, The parameters of the quantized convolutional neural networks for N machine vision tasks are obtained through optimization of bit rate and task accuracy: in, Let λ represent the loss function related to the accuracy of the i-th machine vision task. i >0 is a preset parameter used to balance the bitrate and task accuracy for the i-th machine vision task. i Let ω be the bitrate of the i-th machine vision task. i >0 is a preset parameter used to balance different machine vision tasks.

10. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to execute an image encoding method for machine and human vision as described in any one of claims 1-5, or an image decoding method for machine and human vision as described in claim 6, or an image compression method for machine and human vision as described in any one of claims 7-9.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, this program can be used to perform an image encoding method for machine and human vision as described in any one of claims 1-5, or an image decoding method for machine and human vision as described in claim 6, or an image compression method for machine and human vision as described in any one of claims 7-9.

Images