Adaptive image coding method based on deep learning
Through the adaptive image encoding method based on deep learning, the object detection model GoogLeNet is used to perform adaptive block division and entropy encoding on the image, which solves the problem of feature target integrity and redundancy in the prior art, and achieves more efficient image compression.
Patent Information
- Application Number
- CN202410930307.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-22
AI Technical Summary
The existing image compression and coding methods lack detection and classification of feature targets in the picture during block division, which may lead to the destruction of the integrity of feature targets and cannot effectively reduce the redundancy and correlation of image information.
Based on deep learning technology, the object detection model GoogLeNet is trained. By performing secondary processing and object detection on the image to be encoded, adaptive block division is performed based on the target classification results and the position of the surrounding box, and different prediction methods and entropy encoding methods are used to predict the image blocks. and encoding.
The integrity protection of image targets is achieved, redundancy and correlation between pixels is reduced, compression rate is improved, and the resulting encoded code stream is more efficient.
Smart Images

Figure CN119011877A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an adaptive image encoding method based on deep learning. Background Art
[0002] In recent years, digital cameras have been used more and more widely, involving the fields of medicine, scientific research, consumption, national defense, commerce, etc. As camera sensors develop, their resolution is getting higher and higher, and the requirements for storage space and transmission bandwidth are gradually increasing. Therefore, the compression encoding of video images is becoming more and more important;
[0003] Image compression coding is to transform and combine the image information sources to be processed using certain rules, reduce the redundancy and correlation between image information, and ultimately use as few symbols as possible to represent as much image information as possible; current image compression coding is to divide a single image into blocks, encode each block, and finally obtain a code stream for each block; however, when dividing the blocks, there is a lack of detection and classification of feature targets in the image, which may cause the feature targets to be divided into multiple blocks, and the integrity of the feature targets of the image may be destroyed after decoding; for this reason, the present invention proposes an adaptive image coding method based on deep learning. Summary of the invention
[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an adaptive image coding method based on deep learning.
[0005] To achieve the above object, a first aspect of the present invention provides an adaptive image coding method based on deep learning, comprising the following steps:
[0006] Step 1: Train the target detection model GoogLeNet based on the image dataset; the image dataset contains 60,000 images, including airplanes, ships, people, cars, animals, and landscapes;
[0007] Step 2: performing secondary processing on the image to be encoded; the secondary processing includes but is not limited to graying, edge enhancement, filtering and noise removal;
[0008] Step 3: Input the secondary processed image features into the trained target detection model GoogLeNet to obtain the final target classification result and the target bounding box position;
[0009] Step 4: Divide the image to be encoded into blocks according to the target classification results and the position of the target bounding box; the number of targets is consistent with the number of blocks, and each block contains one target;
[0010] Step 5: Predict each image block separately, perform entropy coding on the prediction residual and prediction mode obtained based on the image block, and finally obtain the code stream of each image block; the specific steps are as follows:
[0011] S51: For a certain image block, let H i,j Represents the grayscale value of the pixel in the i-th row and j-th column of the image block;
[0012] S52: using different prediction methods to predict each image block one by one to obtain a corresponding prediction value; calculating the difference between the prediction value and the corresponding original pixel grayscale value, and then rounding down the residual;
[0013] By comparing the sum of the absolute values of the residuals obtained by various prediction methods, an optimal prediction method that minimizes the sum is selected as the prediction method for the current image block;
[0014] S53: performing entropy coding on the positive integer pixel residuals obtained in step S52 and the optimal prediction method, and finally obtaining a code stream for each block.
[0015] Furthermore, the target detection model GoogLeNet is obtained based on the image dataset training, specifically:
[0016] S11: randomly extract 50,000 images from the image dataset as the training set and 10,000 images as the test set;
[0017] S12: Use the backbone network to extract features from the images in the dataset. For each input image, a 3D feature space is obtained based on feature maps of different lengths, widths, and numbers of channels output by the backbone network at different stages.
[0018] S13: Based on the initial query configuration and decoder, establish the target detection model GoogLeNet;
[0019] The initial query configuration refers to: generating N initial query quantities for initial encoding of the input feature map, the query quantity includes the initial semantic vector F0 and the initial position vector (x, y, z, r) corresponding to the query sampling point;
[0020] S14: The configured network model is trained using the training set and the test set, and the combination of focal loss, L1loss and GIoU loss is used as the loss function. The AdamW optimizer is used to update the network parameters through the back propagation algorithm to obtain the target detection model GoogLeNet with the minimum loss.
[0021] Furthermore, the prediction methods for predicting the image block include the following three methods:
[0022] The first one:
[0023] For H i,j , pixels with j≤1 are not predicted;
[0024] For H i,j , j ≥ 2 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;
[0025] Second type:
[0026] For H i,j , i≤1,j=2,3 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;
[0027] For H i,j , i≤1,j≥4 pixels are predicted, let H i,j Prediction value = (H i,j-2 +H i,j-4 ) / 3;
[0028] For H i,j , i≥2,j≥2 pixels are predicted, let H i,j Prediction value = (H i-2,j +H i,j-2 ) / 3;
[0029] The third type:
[0030] For H i,j , pixels with i=0,j=0 are not predicted;
[0031] For H i,j , i=0,j≥1 pixels are predicted, let H i,j Prediction value = H i,j--1 ;
[0032] For H i,j , i ≥ 1, j = 0 pixels are predicted, let H i,j Prediction value = H i-1,j ;
[0033] For H i,j , i ≥ 1, j ≥ 1 pixels are predicted, let H i,j Prediction value = H i-1,j ×0.3+H i-1,j-1 ×0.3+H i,j-1 ×0.4.
[0034] Furthermore, in the secondary processing, the strategy of using Fourier and Walsh algorithms simultaneously is used.
[0035] Furthermore, (x, y, z) is the coordinate of the sampling point in the 3D feature space, r is the base-2 logarithm of the aspect ratio of the feature map, the initial semantic vector F0 is randomly sampled from the standard normal distribution N(0,1), and the initial position vector (x, y, z, r) is set to cover the entire feature map.
[0036] Furthermore, the input of the decoder is the initial query volume generated by the encoder, and the output is the query volume in the same format after being optimized by the decoder.
[0037] Furthermore, Huffman coding is selected to perform entropy coding on pixel residuals, and exponential Golomb coding is used to perform entropy coding on the optimal prediction method.
[0038] Compared with the prior art, the present invention has the following beneficial effects:
[0039] The present invention obtains a target detection model GoogLeNet through training based on deep learning technology, performs target detection on an image, divides the image into blocks according to the target classification result and the target bounding box position for parallel processing while ensuring the integrity of the image target; then uses the optimal prediction method for each image block to obtain the current pixel prediction value, and then obtains the pixel residual value, reduces the correlation between pixels, and reduces redundancy; finally, uses Huffman coding to perform entropy coding on the prediction residual to obtain the coding code streams of different blocks, thereby achieving the purpose of improving the compression rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 This is a flowchart of an adaptive image coding method based on deep learning in the present invention.
[0042] Figure 2 This is a training flow chart of the target detection model GoogLeNet in the present invention. DETAILED DESCRIPTION
[0043] The technical scheme of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0044] See also Figure 1 to Figure 2 The first aspect of the present invention provides an adaptive image coding method based on deep learning, comprising the following steps:
[0045] Step 1: Train the target detection model GoogLeNet based on the image dataset; specifically include:
[0046] S11: The image dataset contains 60,000 images with 10 categories of labels, including airplanes, ships, people, cars, animals, and landscapes. First, 50,000 images are randomly selected from the dataset as the training set and 10,000 images are selected as the test set.
[0047] S12: Use the backbone network to extract features from the images in the dataset. For each input image, based on the feature maps of different lengths, widths, and numbers of channels output by the backbone network at different stages, a 3D feature space is obtained, in which subsequent sampling processing is completed;
[0048] S13: Based on the initial query configuration and decoder, establish the target detection model GoogLeNet;
[0049] The initial query configuration refers to: generating N initial query quantities for initial encoding of the input feature map, the query quantity includes the initial semantic vector F0 and the initial position vector (x, y, z, r) corresponding to the query sampling point;
[0050] It should be noted that: (x, y, z) is the coordinate of the sampling point in the 3D feature space, r is the base-2 logarithm of the aspect ratio of the feature map, the initial semantic vector F0 is randomly sampled from the standard normal distribution N(0,1), and the initial position vector (x, y, z, r) is set to cover the entire feature map;
[0051] It should be understood that the input of the decoder is the initial query volume generated by the encoder, and the output is the query volume in the same format after being optimized by the decoder;
[0052] S14: Train the configured network model using the training set and the test set, use the combination of focal loss, L1loss and GIoU loss as the loss function, use the AdamW optimizer, and update the network parameters through the back propagation algorithm to obtain the target detection model GoogLeNet with the minimum loss;
[0053] Step 2: Perform secondary processing on the image to be encoded to improve the contrast of the image; the secondary processing includes but is not limited to graying, edge enhancement, filtering and noise removal, and the strategy of using Fourier and Walsh algorithms to process simultaneously during the secondary processing;
[0054] Step 3: Input the secondary processed image features into the trained target detection model GoogLeNet to obtain the final target classification result and the target bounding box position;
[0055] It should be understood that the image feature refers to the feature map obtained by the image to be encoded according to the method in step S12;
[0056] Step 4: Divide the image to be encoded into blocks according to the target classification results and the position of the target bounding box; the number of targets is consistent with the number of blocks, and each block contains one target; these blocks constitute a complete subset of the image and do not overlap with each other;
[0057] Step 5: Predict each image block separately, perform entropy coding on the prediction residual and prediction mode obtained based on the image block, and finally obtain the code stream of each image block; the specific steps are as follows:
[0058] S51: For a certain image block, let H i,j Represents the grayscale value of the pixel in the i-th row and j-th column of the image block;
[0059] S52: using different prediction methods to predict each image block one by one to obtain a corresponding prediction value; calculating the difference between the prediction value and the corresponding original pixel grayscale value, and then rounding down the residual;
[0060] By comparing the sum of the absolute values of the residuals obtained by various prediction methods, an optimal prediction method that minimizes the sum is selected as the prediction method for the current image block;
[0061] It should be noted that: since the residual range is -255 to 255, the residual value less than 0 is added with 256 to make the pixel residual an integer between 0 and 255;
[0062] There are three prediction methods for predicting image blocks:
[0063] The first one:
[0064] For H i,j , pixels with j≤1 are not predicted;
[0065] For H i,j , j ≥ 2 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;
[0066] Second type:
[0067] For H i,j , i≤1,j=2,3 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;
[0068] For H i,j , i≤1,j≥4 pixels are predicted, let H i,j Prediction value = (H i,j-2 +H i,j-4 ) / 3;
[0069] For H i,j , i≥2,j≥2 pixels are predicted, let H i,j Prediction value = (H i-2,j +H i,j-2 ) / 3;
[0070] The third type:
[0071] For H i,j , pixels with i=0,j=0 are not predicted;
[0072] For H i,j , i=0,j≥1 pixels are predicted, let H i,j Prediction value = H i,j--1 ;
[0073] For H i,j , i ≥ 1, j = 0 pixels are predicted, let H i,j Prediction value = H i-1,j ;
[0074] For H i,j , i ≥ 1, j ≥ 1 pixels are predicted, let H i,j Prediction value = H i-1,j ×0.3+H i-1,j-1 ×0.3+H i,j-1 ×0.4;
[0075] S53: Entropy coding is performed on the positive integer pixel residuals and the optimal prediction method obtained in step S52, and finally a code stream of each block is obtained; the present application uses Huffman coding to entropy code the pixel residuals, and uses exponential Golomb coding to entropy code the optimal prediction method;
[0076] This application performs target detection on images based on deep learning technology, divides the image into blocks according to the target classification results and the target bounding box position for parallel processing while ensuring the integrity of the image target; then uses the optimal prediction method for each image block to obtain the current pixel prediction value, and then obtains the pixel residual value, reduces the correlation between pixels, and reduces redundancy; finally, uses Huffman coding to perform entropy coding on the prediction residual to obtain the encoded code stream of different blocks, thereby achieving the purpose of improving the compression rate.
[0077] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.
[0078] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.
Claims
1. An adaptive image coding method based on deep learning, characterized in that: include: Step 1: Train the target detection model GoogLeNet based on the image dataset; the image dataset contains 60,000 images, including airplanes, ships, people, cars, animals, and landscapes; Step 2: Perform secondary processing on the image to be encoded; the secondary processing includes graying, edge enhancement, filtering and noise removal; Step 3: Input the secondary processed image features into the trained target detection model GoogLeNet to obtain the final target classification result and the target bounding box position; Step 4: Divide the image to be encoded into blocks according to the target classification result and the target bounding box position; The number of targets is consistent with the number of blocks, and each block contains one target; Step 5: Predict each image block separately, perform entropy coding on the prediction residual and prediction mode obtained based on the image block, and finally obtain the code stream of each image block; the specific steps are as follows: S51: For a certain image block, let H i,j Represents the grayscale value of the pixel in the i-th row and j-th column of the image block; S52: using different prediction methods to predict each image block one by one to obtain a corresponding prediction value; calculating the difference between the prediction value and the corresponding original pixel grayscale value, and then rounding down the residual; By comparing the sum of the absolute values of the residuals obtained by various prediction methods, an optimal prediction method that minimizes the sum is selected as the prediction method for the current image block; S53: performing entropy coding on the positive integer pixel residuals obtained in step S52 and the optimal prediction method, and finally obtaining a code stream for each block.
2. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: The target detection model GoogLeNet is trained based on the image dataset, including: S11: randomly extract 50,000 images from the image dataset as the training set and 10,000 images as the test set; S12: Use the backbone network to extract features from the images in the dataset. For each input image, a 3D feature space is obtained based on feature maps of different lengths, widths, and numbers of channels output by the backbone network at different stages. S13: Based on the initial query configuration and decoder, establish the target detection model GoogLeNet; The initial query configuration refers to: generating N initial query quantities for initial encoding of the input feature map, the query quantity includes the initial semantic vector F0 and the initial position vector (x, y, z, r) corresponding to the query sampling point; S14: The configured network model is trained using the training set and the test set, using the combination of focalloss, L1loss and GIoUloss as the loss function, using the AdamW optimizer, and updating the network parameters through the back propagation algorithm to obtain the target detection model GoogLeNet with the minimum loss.
3. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: There are three prediction methods for predicting image blocks: The first one: For H i,j , pixels with j≤1 are not predicted; For H i,j , j ≥ 2 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ; Second type: For H i,j , i≤1,j=2,3 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ; For H i,j , i≤1,j≥4 pixels are predicted, let H i,j Prediction value = (H i,j-2 +H i,j-4 ) / 3; For H i,j , i≥2,j≥2 pixels are predicted, let H i,j Prediction value = (H i-2,j +H i,j-2 ) / 3; The third type: For H i,j , pixels with i=0,j=0 are not predicted; For H i,j , i=0,j≥1 pixels are predicted, let H i,j Prediction value = H i,j--1 ; For H i,j , i ≥ 1, j = 0 pixels are predicted, let H i,j Prediction value = H i-1,j ; For H i,j , i ≥ 1, j ≥ 1 pixels are predicted, let H i,j Prediction value = H i-1,j ×0.3+H i-1,j-1 ×0.3+H i,j-1 ×0.
4.
4. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: In the secondary processing, the strategy of using Fourier and Walsh algorithms simultaneously is used.
5. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: (x, y, z) is the coordinate of the sampling point in the 3D feature space, r is the base-2 logarithm of the aspect ratio of the feature map, the initial semantic vector F0 is randomly sampled from the standard normal distribution N(0, 1), and the initial position vector (x, y, z, r) is set to cover the entire feature map.
6. The method of adaptive image coding based on deep learning according to claim 2, characterized in that: The input of the decoder is the initial query volume generated by the encoder, and the output is the query volume in the same format after being optimized by the decoder.
7. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: Huffman coding is used to entropy encode the pixel residual, and exponential Golomb coding is used to entropy encode the optimal prediction method.
