Adaptive image coding method based on deep learning

Through the adaptive image encoding method based on deep learning, the object detection model GoogLeNet is used to perform adaptive block division and entropy encoding on the image, which solves the problem of feature target integrity and redundancy in the prior art, and achieves more efficient image compression.

CN119011877APending Publication Date: 2024-11-22ANHUI RUIJI INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410930307.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing image compression and coding methods lack detection and classification of feature targets in the picture during block division, which may lead to the destruction of the integrity of feature targets and cannot effectively reduce the redundancy and correlation of image information.

Method used

Based on deep learning technology, the object detection model GoogLeNet is trained. By performing secondary processing and object detection on the image to be encoded, adaptive block division is performed based on the target classification results and the position of the surrounding box, and different prediction methods and entropy encoding methods are used to predict the image blocks. and encoding.

Benefits of technology

The integrity protection of image targets is achieved, redundancy and correlation between pixels is reduced, compression rate is improved, and the resulting encoded code stream is more efficient.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011877A_ABST
    Figure CN119011877A_ABST
Patent Text Reader

Abstract

The invention discloses a self-adaptive image coding method based on deep learning, relates to the technical field of image processing, and solves the technical problems that in the current image compression coding process, detection and classification of feature targets are lacked, the integrity of the feature targets is possibly damaged after decoding, and the compression rate is low. Comprising the following steps: training based on an image data set to obtain a target detection model GoogLeNet; inputting the secondarily processed image features into the trained target detection model GoogLeNet to obtain a final target classification result and a target bounding box position; performing block division on the to-be-coded image according to the target classification result and the position of the target bounding box so as to facilitate parallel processing, and ensuring the integrity of the image target at the same time; and then predicting each image block, performing entropy coding on a prediction residual error and a prediction mode obtained based on the image blocks, and finally obtaining a code stream of each image block, thereby achieving the purpose of improving the compression ratio.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an adaptive image encoding method based on deep learning. Background Art

[0002] In recent years, digital cameras have been used more and more widely, involving the fields of medicine, scientific research, consumption, national defense, commerce, etc. As camera sensors develop, their resolution is getting higher and higher, and the requirements for storage space and transmission bandwidth are gradually increasing. Therefore, the compression encoding of video images is becoming more and more important;

[0003] Image compression coding is to transform and combine the image information sources to be processed using certain rules, reduce the redundancy and correlation between image information, and ultimately use as few symbols as possible to represent as much image information as possible; current image compression coding is to divide a single image into blocks, encode each block, and finally obtain a code stream for each block; however, when dividing the blocks, there is a lack of detection and classification of feature targets in the image, which may cause the feature targets to be divided into multiple blocks, and the integrity of the feature targets of the image may be destroyed after decoding; for this reason, the present invention proposes an adaptive image coding method based on deep learning. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an adaptive image coding method based on deep learning.

[0005] To achieve the above object, a first aspect of the present invention provides an adaptive image coding method based on deep learning, comprising the following steps:

[0006] Step 1: Train the target detection model GoogLeNet based on the image dataset; the image dataset contains 60,000 images, including airplanes, ships, people, cars, animals, and landscapes;

[0007] Step 2: performing secondary processing on the image to be encoded; the secondary processing includes but is not limited to graying, edge enhancement, filtering and noise removal;

[0008] Step 3: Input the secondary processed image features into the trained target detection model GoogLeNet to obtain the final target classification result and the target bounding box position;

[0009] Step 4: Divide the image to be encoded into blocks according to the target classification results and the position of the target bounding box; the number of targets is consistent with the number of blocks, and each block contains one target;

[0010] Step 5: Predict each image block separately, perform entropy coding on the prediction residual and prediction mode obtained based on the image block, and finally obtain the code stream of each image block; the specific steps are as follows:

[0011] S51: For a certain image block, let H i,j Represents the grayscale value of the pixel in the i-th row and j-th column of the image block;

[0012] S52: using different prediction methods to predict each image block one by one to obtain a corresponding prediction value; calculating the difference between the prediction value and the corresponding original pixel grayscale value, and then rounding down the residual;

[0013] By comparing the sum of the absolute values ​​of the residuals obtained by various prediction methods, an optimal prediction method that minimizes the sum is selected as the prediction method for the current image block;

[0014] S53: performing entropy coding on the positive integer pixel residuals obtained in step S52 and the optimal prediction method, and finally obtaining a code stream for each block.

[0015] Furthermore, the target detection model GoogLeNet is obtained based on the image dataset training, specifically:

[0016] S11: randomly extract 50,000 images from the image dataset as the training set and 10,000 images as the test set;

[0017] S12: Use the backbone network to extract features from the images in the dataset. For each input image, a 3D feature space is obtained based on feature maps of different lengths, widths, and numbers of channels output by the backbone network at different stages.

[0018] S13: Based on the initial query configuration and decoder, establish the target detection model GoogLeNet;

[0019] The initial query configuration refers to: generating N initial query quantities for initial encoding of the input feature map, the query quantity includes the initial semantic vector F0 and the initial position vector (x, y, z, r) corresponding to the query sampling point;

[0020] S14: The configured network model is trained using the training set and the test set, and the combination of focal loss, L1loss and GIoU loss is used as the loss function. The AdamW optimizer is used to update the network parameters through the back propagation algorithm to obtain the target detection model GoogLeNet with the minimum loss.

[0021] Furthermore, the prediction methods for predicting the image block include the following three methods:

[0022] The first one:

[0023] For H i,j , pixels with j≤1 are not predicted;

[0024] For H i,j , j ≥ 2 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;

[0025] Second type:

[0026] For H i,j , i≤1,j=2,3 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;

[0027] For H i,j , i≤1,j≥4 pixels are predicted, let H i,j Prediction value = (H i,j-2 +H i,j-4 ) / 3;

[0028] For H i,j , i≥2,j≥2 pixels are predicted, let H i,j Prediction value = (H i-2,j +H i,j-2 ) / 3;

[0029] The third type:

[0030] For H i,j , pixels with i=0,j=0 are not predicted;

[0031] For H i,j , i=0,j≥1 pixels are predicted, let H i,j Prediction value = H i,j--1 ;

[0032] For H i,j , i ≥ 1, j = 0 pixels are predicted, let H i,j Prediction value = H i-1,j ;

[0033] For H i,j , i ≥ 1, j ≥ 1 pixels are predicted, let H i,j Prediction value = H i-1,j ×0.3+H i-1,j-1 ×0.3+H i,j-1 ×0.4.

[0034] Furthermore, in the secondary processing, the strategy of using Fourier and Walsh algorithms simultaneously is used.

[0035] Furthermore, (x, y, z) is the coordinate of the sampling point in the 3D feature space, r is the base-2 logarithm of the aspect ratio of the feature map, the initial semantic vector F0 is randomly sampled from the standard normal distribution N(0,1), and the initial position vector (x, y, z, r) is set to cover the entire feature map.

[0036] Furthermore, the input of the decoder is the initial query volume generated by the encoder, and the output is the query volume in the same format after being optimized by the decoder.

[0037] Furthermore, Huffman coding is selected to perform entropy coding on pixel residuals, and exponential Golomb coding is used to perform entropy coding on the optimal prediction method.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention obtains a target detection model GoogLeNet through training based on deep learning technology, performs target detection on an image, divides the image into blocks according to the target classification result and the target bounding box position for parallel processing while ensuring the integrity of the image target; then uses the optimal prediction method for each image block to obtain the current pixel prediction value, and then obtains the pixel residual value, reduces the correlation between pixels, and reduces redundancy; finally, uses Huffman coding to perform entropy coding on the prediction residual to obtain the coding code streams of different blocks, thereby achieving the purpose of improving the compression rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 This is a flowchart of an adaptive image coding method based on deep learning in the present invention.

[0042] Figure 2 This is a training flow chart of the target detection model GoogLeNet in the present invention. DETAILED DESCRIPTION

[0043] The technical scheme of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1 to Figure 2 The first aspect of the present invention provides an adaptive image coding method based on deep learning, comprising the following steps:

[0045] Step 1: Train the target detection model GoogLeNet based on the image dataset; specifically include:

[0046] S11: The image dataset contains 60,000 images with 10 categories of labels, including airplanes, ships, people, cars, animals, and landscapes. First, 50,000 images are randomly selected from the dataset as the training set and 10,000 images are selected as the test set.

[0047] S12: Use the backbone network to extract features from the images in the dataset. For each input image, based on the feature maps of different lengths, widths, and numbers of channels output by the backbone network at different stages, a 3D feature space is obtained, in which subsequent sampling processing is completed;

[0048] S13: Based on the initial query configuration and decoder, establish the target detection model GoogLeNet;

[0049] The initial query configuration refers to: generating N initial query quantities for initial encoding of the input feature map, the query quantity includes the initial semantic vector F0 and the initial position vector (x, y, z, r) corresponding to the query sampling point;

[0050] It should be noted that: (x, y, z) is the coordinate of the sampling point in the 3D feature space, r is the base-2 logarithm of the aspect ratio of the feature map, the initial semantic vector F0 is randomly sampled from the standard normal distribution N(0,1), and the initial position vector (x, y, z, r) is set to cover the entire feature map;

[0051] It should be understood that the input of the decoder is the initial query volume generated by the encoder, and the output is the query volume in the same format after being optimized by the decoder;

[0052] S14: Train the configured network model using the training set and the test set, use the combination of focal loss, L1loss and GIoU loss as the loss function, use the AdamW optimizer, and update the network parameters through the back propagation algorithm to obtain the target detection model GoogLeNet with the minimum loss;

[0053] Step 2: Perform secondary processing on the image to be encoded to improve the contrast of the image; the secondary processing includes but is not limited to graying, edge enhancement, filtering and noise removal, and the strategy of using Fourier and Walsh algorithms to process simultaneously during the secondary processing;

[0054] Step 3: Input the secondary processed image features into the trained target detection model GoogLeNet to obtain the final target classification result and the target bounding box position;

[0055] It should be understood that the image feature refers to the feature map obtained by the image to be encoded according to the method in step S12;

[0056] Step 4: Divide the image to be encoded into blocks according to the target classification results and the position of the target bounding box; the number of targets is consistent with the number of blocks, and each block contains one target; these blocks constitute a complete subset of the image and do not overlap with each other;

[0057] Step 5: Predict each image block separately, perform entropy coding on the prediction residual and prediction mode obtained based on the image block, and finally obtain the code stream of each image block; the specific steps are as follows:

[0058] S51: For a certain image block, let H i,j Represents the grayscale value of the pixel in the i-th row and j-th column of the image block;

[0059] S52: using different prediction methods to predict each image block one by one to obtain a corresponding prediction value; calculating the difference between the prediction value and the corresponding original pixel grayscale value, and then rounding down the residual;

[0060] By comparing the sum of the absolute values ​​of the residuals obtained by various prediction methods, an optimal prediction method that minimizes the sum is selected as the prediction method for the current image block;

[0061] It should be noted that: since the residual range is -255 to 255, the residual value less than 0 is added with 256 to make the pixel residual an integer between 0 and 255;

[0062] There are three prediction methods for predicting image blocks:

[0063] The first one:

[0064] For H i,j , pixels with j≤1 are not predicted;

[0065] For H i,j , j ≥ 2 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;

[0066] Second type:

[0067] For H i,j , i≤1,j=2,3 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ;

[0068] For H i,j , i≤1,j≥4 pixels are predicted, let H i,j Prediction value = (H i,j-2 +H i,j-4 ) / 3;

[0069] For H i,j , i≥2,j≥2 pixels are predicted, let H i,j Prediction value = (H i-2,j +H i,j-2 ) / 3;

[0070] The third type:

[0071] For H i,j , pixels with i=0,j=0 are not predicted;

[0072] For H i,j , i=0,j≥1 pixels are predicted, let H i,j Prediction value = H i,j--1 ;

[0073] For H i,j , i ≥ 1, j = 0 pixels are predicted, let H i,j Prediction value = H i-1,j ;

[0074] For H i,j , i ≥ 1, j ≥ 1 pixels are predicted, let H i,j Prediction value = H i-1,j ×0.3+H i-1,j-1 ×0.3+H i,j-1 ×0.4;

[0075] S53: Entropy coding is performed on the positive integer pixel residuals and the optimal prediction method obtained in step S52, and finally a code stream of each block is obtained; the present application uses Huffman coding to entropy code the pixel residuals, and uses exponential Golomb coding to entropy code the optimal prediction method;

[0076] This application performs target detection on images based on deep learning technology, divides the image into blocks according to the target classification results and the target bounding box position for parallel processing while ensuring the integrity of the image target; then uses the optimal prediction method for each image block to obtain the current pixel prediction value, and then obtains the pixel residual value, reduces the correlation between pixels, and reduces redundancy; finally, uses Huffman coding to perform entropy coding on the prediction residual to obtain the encoded code stream of different blocks, thereby achieving the purpose of improving the compression rate.

[0077] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0078] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An adaptive image coding method based on deep learning, characterized in that: include: Step 1: Train the target detection model GoogLeNet based on the image dataset; the image dataset contains 60,000 images, including airplanes, ships, people, cars, animals, and landscapes; Step 2: Perform secondary processing on the image to be encoded; the secondary processing includes graying, edge enhancement, filtering and noise removal; Step 3: Input the secondary processed image features into the trained target detection model GoogLeNet to obtain the final target classification result and the target bounding box position; Step 4: Divide the image to be encoded into blocks according to the target classification result and the target bounding box position; The number of targets is consistent with the number of blocks, and each block contains one target; Step 5: Predict each image block separately, perform entropy coding on the prediction residual and prediction mode obtained based on the image block, and finally obtain the code stream of each image block; the specific steps are as follows: S51: For a certain image block, let H i,j Represents the grayscale value of the pixel in the i-th row and j-th column of the image block; S52: using different prediction methods to predict each image block one by one to obtain a corresponding prediction value; calculating the difference between the prediction value and the corresponding original pixel grayscale value, and then rounding down the residual; By comparing the sum of the absolute values ​​of the residuals obtained by various prediction methods, an optimal prediction method that minimizes the sum is selected as the prediction method for the current image block; S53: performing entropy coding on the positive integer pixel residuals obtained in step S52 and the optimal prediction method, and finally obtaining a code stream for each block.

2. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: The target detection model GoogLeNet is trained based on the image dataset, including: S11: randomly extract 50,000 images from the image dataset as the training set and 10,000 images as the test set; S12: Use the backbone network to extract features from the images in the dataset. For each input image, a 3D feature space is obtained based on feature maps of different lengths, widths, and numbers of channels output by the backbone network at different stages. S13: Based on the initial query configuration and decoder, establish the target detection model GoogLeNet; The initial query configuration refers to: generating N initial query quantities for initial encoding of the input feature map, the query quantity includes the initial semantic vector F0 and the initial position vector (x, y, z, r) corresponding to the query sampling point; S14: The configured network model is trained using the training set and the test set, using the combination of focalloss, L1loss and GIoUloss as the loss function, using the AdamW optimizer, and updating the network parameters through the back propagation algorithm to obtain the target detection model GoogLeNet with the minimum loss.

3. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: There are three prediction methods for predicting image blocks: The first one: For H i,j , pixels with j≤1 are not predicted; For H i,j , j ≥ 2 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ; Second type: For H i,j , i≤1,j=2,3 pixels are predicted, let H i,j Predicted value = 0.8 × H i,j-2 ; For H i,j , i≤1,j≥4 pixels are predicted, let H i,j Prediction value = (H i,j-2 +H i,j-4 ) / 3; For H i,j , i≥2,j≥2 pixels are predicted, let H i,j Prediction value = (H i-2,j +H i,j-2 ) / 3; The third type: For H i,j , pixels with i=0,j=0 are not predicted; For H i,j , i=0,j≥1 pixels are predicted, let H i,j Prediction value = H i,j--1 ; For H i,j , i ≥ 1, j = 0 pixels are predicted, let H i,j Prediction value = H i-1,j ; For H i,j , i ≥ 1, j ≥ 1 pixels are predicted, let H i,j Prediction value = H i-1,j ×0.3+H i-1,j-1 ×0.3+H i,j-1 ×0.

4.

4. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: In the secondary processing, the strategy of using Fourier and Walsh algorithms simultaneously is used.

5. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: (x, y, z) is the coordinate of the sampling point in the 3D feature space, r is the base-2 logarithm of the aspect ratio of the feature map, the initial semantic vector F0 is randomly sampled from the standard normal distribution N(0, 1), and the initial position vector (x, y, z, r) is set to cover the entire feature map.

6. The method of adaptive image coding based on deep learning according to claim 2, characterized in that: The input of the decoder is the initial query volume generated by the encoder, and the output is the query volume in the same format after being optimized by the decoder.

7. The method of adaptive image coding based on deep learning according to claim 1, characterized in that: Huffman coding is used to entropy encode the pixel residual, and exponential Golomb coding is used to entropy encode the optimal prediction method.