Image compression method and device based on deep learning

Through deep learning technology, image frames are hierarchically compressed, and features are extracted and reconstructed using multi-layer convolutional neural networks, combined with compression ratio coefficients, the problem of redundant features is difficult to remove in the existing technology, and efficient image compression and Data transmission.

CN113379858BActive Publication Date: 2024-11-05AI SUPER EYE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110597919.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-05-31
Publication Date
2024-11-05
Estimated Expiration
2041-05-31

AI Technical Summary

Technical Problem

The prior art is difficult to effectively remove redundant data features during image compression, resulting in low compression ratio and insufficient general applicability to different application scenarios and image smoothness.

Method used

Through a deep learning-based method, the target category and detection frame information in the image frame to be detected are obtained, and feature extraction and reconstruction are used for multi-layer convolutional neural network, and hierarchical compression is performed with predefined compression ratio coefficients to realize image frames hierarchical compression of different target areas.

Benefits of technology

It significantly improves the compression ratio of the image, effectively removes redundant data features, improves the storage and transmission efficiency of data, and selectively blurs the specified targets, protects user privacy, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113379858B_ABST
    Figure CN113379858B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides an image compression method and device based on deep learning, the method comprising: obtaining an image frame to be detected, detecting each target to be detected in the image frame, obtaining the category and detection frame information of each target to be detected, extracting features from the image frame through a multi-layer convolutional neural network, and obtaining a first feature representation vector of the image frame; uniformly quantizing the first feature representation vector to obtain a second feature representation vector, and reconstructing features from the second feature representation vector; performing distortion calculation on the image frame according to predefined compression ratio coefficients of each category; compressing the image frame after the distortion calculation, and obtaining images of each target to be detected in the image frame compressed according to their respective compression ratios. Through the present invention, hierarchical compression of different targets in the image to be detected is achieved, redundant data features are effectively removed, and the compression ratio of the image is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and image compression, and in particular to an image compression method and device based on deep learning. Background Art

[0002] In recent years, high-position video technology has developed rapidly. This technology can be used to detect and capture parking status, vehicle parking, vehicle violations, etc., and can record the entire process in the form of pictures and videos, while forming a complete parking or violation evidence data chain, which greatly alleviates the previous phenomenon of random charges for roadside parking. In this process, the storage and transmission of data captured by high-position cameras also brings major challenges. How to reduce redundant information in image data so as to store and transmit data more efficiently has become an urgent problem to be solved. In the prior art, on the one hand, by combining edge computing and deep learning with intelligent terminal image compression algorithms, image compression models are deployed in the process of terminal and cloud data transmission to reduce the amount of network transmission data. However, this method can only compress the global image according to a unified compression ratio, which may easily cause some areas to be over-compressed and blurred, while some areas still have a large number of redundant features; on the other hand, multi-layer sparse automatic coding and clustering algorithms are used to compress images, but this method is limited to experimental verification of grayscale images, which is not generally applicable to current application scenarios, and is likely to cause the compressed image to be non-smooth; on the other hand, by using convolutional neural networks to extract semantically important areas of the image, the image is compressed hierarchically according to the semantic importance of each area, but the semantic importance of the network analysis in this method often has large errors, resulting in the inability to obtain an effective compressed image. Summary of the invention

[0003] The embodiments of the present invention provide an image compression method and device based on deep learning, which realizes hierarchical compression of different targets in the image to be detected, effectively removes redundant data features, and greatly improves the compression ratio of the image.

[0004] On the one hand, an embodiment of the present invention provides an image compression method based on deep learning, comprising:

[0005] Acquire an image frame to be detected, detect each target to be detected in the image frame, obtain the category and detection frame information of each target to be detected, and extract features of the image frame through a multi-layer convolutional neural network according to the category and detection frame information of each target to be detected, so as to obtain a first feature representation vector of the image frame;

[0006] Uniformly quantizing the first feature representation vector to obtain a second feature representation vector, and reconstructing the second feature representation vector through a multi-layer convolutional neural network;

[0007] Based on the reconstructed second feature representation vector, the first feature representation vector and the detection frame information of each target to be detected, and according to the predefined compression ratio coefficients of each category, the distortion calculation is performed on the image frame;

[0008] The image frame after distortion calculation is compressed to obtain images of each target to be detected in the image frame after being compressed according to respective compression ratios.

[0009] Furthermore, the acquiring of the image frame to be detected, detecting each target to be detected in the image frame, and obtaining the category and detection frame information of each target to be detected includes:

[0010] Acquire the image frame to be detected based on a predetermined frequency;

[0011] Detecting each target to be detected in the image frame by using a predetermined target detection algorithm to obtain a detection result including a category and detection frame information of each target to be detected;

[0012] The detection frame information includes pixel coordinate information of each vertex of the detection frame.

[0013] Furthermore, the extracting features of the image frame by a multi-layer convolutional neural network according to the category and detection frame information of each target to be detected to obtain a first feature representation vector of the image frame includes:

[0014] According to the category and detection frame information of each target to be detected, the image frame is input into a multi-layer convolutional neural network with a predetermined first pixel size;

[0015] Extracting a first feature representation vector of the image frame of a second pixel size through encoding operations of multiple convolutional layers and generalized normalization layers;

[0016] The first pixel size is larger than the second pixel size.

[0017] Furthermore, uniformly quantizing the first feature representation vector to obtain a second feature representation vector includes:

[0018] The first feature representation vector of a second pixel size is uniformly quantized to obtain a second feature representation vector of a second pixel size after uniform quantization.

[0019] Furthermore, before the step of uniformly quantizing the first feature representation vector to obtain the second feature representation vector, the method further comprises:

[0020] The uniform quantization model is obtained by training with the addition of uniform noise.

[0021] Furthermore, the reconstructing the second feature representation vector by using a multi-layer convolutional neural network includes:

[0022] Inputting the second feature representation vector into a multi-layer convolutional neural network;

[0023] Through decoding operations of multiple convolutional layers and generalized normalization layers, the size of the second feature representation vector of the second pixel size is restored to the first pixel size.

[0024] Furthermore, the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected are used to perform distortion calculation on the image frame according to predefined compression ratio coefficients of various categories, including:

[0025] Determine regional features of different regions in each target to be detected according to the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected;

[0026] Determine the category of each target to be detected, and determine the compression ratio coefficient of each target to be detected according to the predefined compression ratio coefficients of each category;

[0027] The image frame is subjected to distortion calculation by a predetermined image quality evaluation function according to the regional features of each target to be detected and the compression ratio coefficient.

[0028] Furthermore, compressing the image frame after the distortion calculation to obtain an image of each target to be detected in the image frame compressed according to its own compression ratio includes:

[0029] Determining the compression bit rate of the image frame by cross entropy;

[0030] Based on the compression code rate, the image frame after the distortion calculation is compressed to obtain an image of each target to be detected in the image frame after being compressed according to its own compression ratio.

[0031] On the other hand, an embodiment of the present invention provides an image compression device based on deep learning, comprising:

[0032] A detection and feature extraction module, used to obtain an image frame to be detected, detect each target to be detected in the image frame, obtain the category and detection frame information of each target to be detected, and extract features of the image frame through a multi-layer convolutional neural network according to the category and detection frame information of each target to be detected, so as to obtain a first feature representation vector of the image frame;

[0033] A feature reconstruction module, used for uniformly quantizing the first feature representation vector to obtain a second feature representation vector, and reconstructing the second feature representation vector through a multi-layer convolutional neural network;

[0034] A distortion calculation module, configured to perform distortion calculation on the image frame according to predefined compression ratio coefficients of various categories based on the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected;

[0035] The compression module is used to compress the image frame after the distortion calculation to obtain an image of each target to be detected in the image frame after being compressed according to its own compression ratio.

[0036] Furthermore, the detection and feature extraction module includes:

[0037] An acquisition unit, used for acquiring an image frame to be detected based on a predetermined frequency;

[0038] A detection unit, configured to detect each target to be detected in the image frame by using a predetermined target detection algorithm, and obtain a detection result including a category and detection frame information of each target to be detected;

[0039] The detection frame information includes pixel coordinate information of each vertex of the detection frame.

[0040] Furthermore, the detection and feature extraction module includes:

[0041] A first input unit, configured to input the image frame into a multi-layer convolutional neural network with a predetermined first pixel size according to the category and detection frame information of each target to be detected;

[0042] An extraction unit, configured to extract a first feature representation vector of the image frame of a second pixel size through encoding operations of a plurality of convolutional layers and a generalized normalization layer;

[0043] The first pixel size is larger than the second pixel size.

[0044] Furthermore, the feature reconstruction module includes:

[0045] The uniform quantization unit is used to uniformly quantize the first feature representation vector of the second pixel size to obtain a second feature representation vector of the second pixel size after uniform quantization.

[0046] Further, including:

[0047] The training module is used to approximate quantization by adding uniform noise to the training method, and obtain a uniform quantization model through training.

[0048] Furthermore, the feature reconstruction module includes:

[0049] A second input unit, used to input the second feature representation vector into the multi-layer convolutional neural network;

[0050] The restoration unit is used to restore the size of the second feature representation vector of the second pixel size to the first pixel size through decoding operations of multiple convolutional layers and generalized normalization layers.

[0051] Furthermore, the distortion calculation module is specifically used for

[0052] Determine regional features of different regions in each target to be detected according to the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected;

[0053] Determine the category of each target to be detected, and determine the compression ratio coefficient of each target to be detected according to the predefined compression ratio coefficients of each category;

[0054] The image frame is subjected to distortion calculation by a predetermined image quality evaluation function according to the regional features of each target to be detected and the compression ratio coefficient.

[0055] Furthermore, the compression module is specifically used for

[0056] Determining the compression bit rate of the image frame by cross entropy;

[0057] Based on the compression code rate, the image frame after the distortion calculation is compressed to obtain an image of each target to be detected in the image frame after being compressed according to its own compression ratio.

[0058] The above technical scheme has the following beneficial effects: through the present invention, target detection is performed on each feature area of ​​the collected image data using a target detection algorithm to obtain the detection frame position and category information of different targets to be detected, and the image frame and the detection frame information of different targets to be detected are sent to the image compression network for learning, and the compression rate of the feature areas of different targets to be detected is set in a graded manner, thereby realizing graded image compression of the image frame, effectively removing redundant data features, and greatly improving the image compression ratio, so that data can be efficiently stored and transmitted. Furthermore, by selectively blurring the specified target to be detected, the user's privacy can be protected and the user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 This is a flow chart of an image compression method based on deep learning in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the structure of an image compression device based on deep learning in one embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] The above technical scheme of the embodiment of the present invention has the following beneficial effects: through the present invention, target detection is performed on each feature area of ​​the collected image data using a target detection algorithm to obtain the detection frame position and category information of different targets to be detected, and the image frame and the detection frame information of different targets to be detected are sent to the image compression network for learning, and the compression rate of the feature areas of different targets to be detected is set in a graded manner, thereby realizing graded image compression of the image frame, effectively removing redundant data features, and greatly improving the compression ratio of the image, so that data can be efficiently stored and transmitted. Furthermore, by selectively blurring the specified target to be detected, the user's privacy can be protected and the user experience is improved.

[0064] The above technical solution of the embodiment of the present invention is described in detail below with reference to application examples:

[0065] The application example of the present invention aims to perform hierarchical compression on different targets in an image to be detected, effectively remove redundant data features, and greatly improve the compression ratio of the image.

[0066] In one possible implementation, step 101 obtains an image frame to be detected, detects each target to be detected in the image frame, obtains the category and detection frame information of each target to be detected, and extracts features of the image frame through a multi-layer convolutional neural network according to the category and detection frame information of each target to be detected, to obtain a first feature representation vector of the image frame, including: obtaining an image frame to be detected based on a predetermined frequency; detecting each target to be detected in the image frame through a predetermined target detection algorithm, to obtain a detection result including the category and detection frame information of each target to be detected; inputting the image frame into a multi-layer convolutional neural network with a predetermined first pixel size according to the category and detection frame information of each target to be detected; and extracting the first feature representation vector of the image frame of a second pixel size through encoding operations of multiple convolutional layers and generalized normalization layers.

[0067] Among them, the predetermined target detection algorithm includes at least one of the single-stage, two-stage and non-anchor frame target detection algorithms anchor-free; the detection frame information includes pixel coordinate information of each vertex of the detection frame, and the first pixel size is larger than the second pixel size.

[0068] For example, in the process of image compression, firstly, an image frame to be detected captured by a video device is obtained based on a predetermined frequency; each target to be detected in the image frame is detected by a predetermined target detection algorithm, such as a single-stage target detection algorithm, a two-stage target detection algorithm, and an anchor-free 2D plane target algorithm to obtain a detection result, wherein the detection result includes the category of each target to be detected, such as a pedestrian, a vehicle, and other categories and detection frame information, such as pixel coordinate information of each vertex of the detection frame of the detection target, wherein the detection result of each frame of the image is stored in a file, and the format of the detection result file includes but is not limited to text files in formats such as xml and json; then, according to the category of each target to be detected and the detection frame information, the image frame is input into a multi-layer convolutional neural network with a predetermined first pixel size, such as 256×256 pixels, and the multi-layer convolutional neural network is used as a main encoder to generate a first feature representation vector; specifically, after four downsampling and four generalized normalization layer encoding operations, a first feature representation vector with a size of 16×16 pixels is obtained. Among them, the convolutional neural network includes convolutional neural network structures such as VGG (Visual Geometry Group Network, VGG model), AlexNet (a convolutional neural network), and ResNet (Residual Network); the generalized normalization layer in this embodiment is more suitable for image compression technology, which can avoid the introduction of noise and make the generated image clearer.

[0069] Through this embodiment, based on the multi-layer neural network structure, the deep learning model has better nonlinear mapping capabilities and can learn the deep-level features of image data, providing an important prerequisite for subsequent image hierarchical compression.

[0070] It should be noted that the images collected by the video device in the embodiment of the present invention cover real road traffic scenes, and can monitor and capture images under different scenes, different viewing angles, and different light intensities. Those skilled in the art can understand that target detection is one of the important issues in the field of computer vision, and is the basis for completing complex visual tasks such as target tracking and scene understanding. It is widely used in video surveillance, intelligent transportation and other fields. The task of target detection is to determine whether there is a target belonging to a specified category in the image, and if so, give the category and location information of the target. Due to the large changes in the category, number, position, scale and appearance of the target in the image, in general, the target detection task is more challenging than the target classification task. The target detection algorithm based on deep learning is divided into two types: a two-stage detection algorithm and a single-stage detection algorithm. The two-stage detection algorithm is also called a target detection algorithm based on candidate regions, which decomposes the target detection process into three steps: candidate region extraction, candidate region classification, and candidate region coordinate correction. The single-stage detection algorithm is also called a target detection algorithm based on regression analysis, which regards the target detection problem as a regression analysis problem of target position and category information, and can directly output the detection result through a neural network model.

[0071] In one possible implementation, step 102 uniformly quantizes the first feature representation vector to obtain a second feature representation vector, and reconstructs the second feature representation vector through a multi-layer convolutional neural network, including: uniformly quantizing the first feature representation vector of a second pixel size to obtain a second feature representation vector of a second pixel size after uniform quantization; inputting the second feature representation vector into the multi-layer convolutional neural network; and restoring the size of the second feature representation vector of the second pixel size to the first pixel size through decoding operations of multiple convolutional layers and generalized normalization layers.

[0072] Before uniformly quantizing the first feature representation vector to obtain the second feature representation vector, the method includes: performing approximate quantization by training with adding uniform noise to obtain a uniform quantization model.

[0073] For example, in the process of image compression, first, a uniform quantization model is obtained by training by adding a calculation method of uniform noise and rounding off the calculation result. Specifically, in the process of training the uniform quantization model, the method of adding uniform noise is adopted to make it approximately differentiable for back propagation optimization. This method avoids the problem that the optimization process of the model cannot be performed due to the non-differentiability caused by the quantization calculation. In the reasoning process of the uniform quantization model, rounding off is performed. After extracting the first feature representation vector of the image frame to be detected of the second pixel size, such as 16×16 pixels, the 16×16 pixels are The first feature representation vector of size is uniformly quantized through a uniform quantization model to obtain a second feature representation vector of size 16×16 pixels after uniform quantization; the second feature representation vector is input into a multi-layer convolutional neural network; through decoding operations of multiple convolutional layers and generalized normalization layers, the size of the second feature representation vector of size 16×16 pixels is restored to 256×256 pixels, and the image is restored to the size of the original image, so that the image is still the original image size after compression, thereby being able to restore the captured content to the greatest extent and avoid the deformation and distortion of the image content caused by the size change during the compression process. Among them, the convolutional neural network in this embodiment includes but is not limited to convolutional neural network structures such as VGG, AlexNet, and ResNet; the anti-generalized normalization layer in this embodiment is more suitable for image compression technology, which can avoid the introduction of noise and generate a clearer image.

[0074] Through this embodiment, the first feature representation vector is uniformly quantized, which can reduce redundant spatial features in the image, thereby reducing the bit rate and greatly reducing the storage space of data.

[0075] In a possible implementation, step 103 performs distortion calculation on the image frame based on the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected according to predefined compression ratio coefficients of each category, including: determining regional features of different regions in each target to be detected according to the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected; determining the category of each target to be detected, and determining the compression ratio coefficient of each target to be detected according to the predefined compression ratio coefficients of each category; performing distortion calculation on the image frame according to the regional features and compression ratio coefficients of each target to be detected through a predetermined image quality evaluation function.

[0076] Among them, step 104 compresses the image frame after the distortion calculation to obtain images of each target to be detected in the image frame after compression according to their respective compression ratios, including: determining the compression bit rate of the image frame by cross entropy; based on the compression bit rate, compressing the image frame after the distortion calculation to obtain images of each target to be detected in the image frame after compression according to their respective compression ratios.

[0077] For example, in the process of image compression, continuing with the above example, according to the second feature representation vector after feature reconstruction, the first feature representation vector and the coordinate information of each vertex of the detection box of each target to be detected, the loss function of the image compression model training is constructed, and the loss function is:

[0078] l=λ*D+R

[0079] Among them, L is the loss function, R represents the compression bit rate of the image, which can be expressed by BPP (bit per pixel, pixel depth). The smaller the BPP value, the lower the compression rate, and vice versa. The choice of bit rate can be controlled by the λ parameter. The larger the λ parameter, the smaller the image distortion reconstructed by the trained model, and the higher the bit rate, and vice versa. D represents the image distortion calculation.

[0080] Then, the regional features of different areas in each target to be detected are determined. For example, the area in the target to be detected can be divided into two areas, the target detection area and other areas outside the target detection area. The category of each target to be detected is determined, and the compression ratio coefficient of each target area to be detected is determined according to the predefined compression ratio coefficients of each category. The image distortion calculation D is:

[0081] D=αD1+(1-α)D2

[0082] Wherein, D1 represents the characteristics of the target detection area, D2 represents the characteristics of other areas outside the target detection area, and α represents the compression ratio coefficient of the characteristics of the target detection area and other areas outside the target detection area. When the compression ratio coefficients of various categories are predefined, they can be set according to the importance of each category. Then, through a predetermined image quality evaluation function, according to the regional characteristics and compression ratio coefficients of each target to be detected, the distortion calculation of the image frame to be detected is performed. The calculation formulas of D1 and D2 are as follows:

[0083]

[0084]

[0085] Among them, x represents the input feature, Represents the reconstruction features, d can use different image quality evaluation functions such as MSE (Mean Square Error), PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity), MS-SSIM (Multi-scale Structural Similarity), etc.

[0086] Then, the compression bit rate of the image frame to be detected is determined by cross entropy as follows:

[0087]

[0088] Among them, y represents the sample, E represents the expectation, and p is the probability distribution function.

[0089] Those skilled in the art can understand that cross entropy is mainly used to measure the difference information between two probability distributions. In this embodiment, the amount of information is calculated by cross entropy to obtain the compression bit rate.

[0090] Specifically, the loss function L is a rate-distortion function, including two parts: the distortion of the original image and the reconstructed image and the compression code rate of the overall framework. The code rate can be selected and controlled by adjusting the coefficient parameters λ of the two. The larger the parameter, the smaller the distortion of the reconstructed image of the trained image compression model, and the smaller the parameter, the larger the distortion of the reconstructed image of the trained model. Reconstructed images with different compression ratios can be obtained by adjusting the parameters. Subsequently, based on the compression code rate, the image frame to be detected after the distortion calculation is compressed to obtain images of each target to be detected in the image frame to be detected after compression according to their respective compression ratios. Specifically, the distortion of the original image and the reconstructed image is calculated using an image quality assessment function calculated pixel by pixel. When calculating the distortion, different coefficient parameters α are set for the target detection area and other areas. Increasing the coefficient of the target detection area increases the image reconstruction quality of the target detection area, while for other areas, the quality of feature reconstruction is reduced, that is, the compression ratio is increased.

[0091] Specifically, the loss function L is:

[0092]

[0093] According to the loss function L and the gradient descent algorithm, the gradient is calculated and the image compression model parameters are updated; the image potential features are output through the feature extraction neural network, and the reconstructed image features are output through the encoder and quantization decoder. Among them, the feature extraction network includes but is not limited to ResNet, VGG, AlexNet and other feature extraction networks.

[0094] Through this embodiment, a deep learning method is used to perform graded image compression on the image frames to be detected in the high-bit video. A lower compression rate is used for the main feature area, which can retain the features of the target area in the image as much as possible. A higher compression rate is used for other secondary feature areas, which can remove redundant features as much as possible, thereby improving the image compression ratio and greatly improving the data storage and transmission efficiency.

[0095] An embodiment of the present invention provides an image compression device based on deep learning, which can implement the method embodiment provided above. For specific function implementation, please refer to the description in the method embodiment, which will not be repeated here.

[0096] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0097] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0098] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0099] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0100] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0101] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0102] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0103] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and blue-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. The above combination can also be included in computer readable media.

[0104] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deep learning-based image compression method, characterized in that: include: Acquire an image frame to be detected, detect each target to be detected in the image frame, obtain the category and detection frame information of each target to be detected, and extract features of the image frame through a multi-layer convolutional neural network according to the category and detection frame information of each target to be detected, so as to obtain a first feature representation vector of the image frame; Uniformly quantizing the first feature representation vector to obtain a second feature representation vector, and reconstructing the second feature representation vector through a multi-layer convolutional neural network; Based on the reconstructed second feature representation vector, the first feature representation vector and the detection frame information of each target to be detected, and according to the predefined compression ratio coefficients of each category, the distortion calculation is performed on the image frame; The uniformly quantizing the first feature representation vector to obtain a second feature representation vector includes: Uniformly quantizing the first feature representation vector of a second pixel size to obtain a uniformly quantized second feature representation vector of a second pixel size; Before the step of uniformly quantizing the first feature representation vector to obtain the second feature representation vector, the method includes: The uniform quantization model is obtained by training with rounded quantization by adding uniform noise. The step of reconstructing the second feature representation vector by using a multi-layer convolutional neural network includes: Inputting the second feature representation vector into a multi-layer convolutional neural network; Through decoding operations of multiple convolutional layers and generalized normalization layers, the size of the second feature representation vector of the second pixel size is restored to the first pixel size; The method of performing distortion calculation on the image frame based on the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected according to the compression ratio coefficients of predefined categories includes: Determine regional features of different regions in each target to be detected according to the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected; Determine the category of each target to be detected, and determine the compression ratio coefficient of each target to be detected according to the predefined compression ratio coefficients of each category; Through a predetermined image quality evaluation function, the image frame is subjected to distortion calculation according to the regional features of each target to be detected and the compression ratio coefficient; the image frame after the distortion calculation is compressed to obtain an image of each target to be detected in the image frame after being compressed according to its own compression ratio.

2. The image compression method according to claim 1, characterized in that: The step of acquiring an image frame to be detected, detecting each target to be detected in the image frame, and obtaining the category and detection frame information of each target to be detected includes: Acquire the image frame to be detected based on a predetermined frequency; Detecting each target to be detected in the image frame by using a predetermined target detection algorithm to obtain a detection result including a category and detection frame information of each target to be detected; The detection frame information includes pixel coordinate information of each vertex of the detection frame.

3. The image compression method according to claim 2, characterized in that: The method of extracting features of the image frame by a multi-layer convolutional neural network according to the category and detection frame information of each target to be detected to obtain a first feature representation vector of the image frame includes: According to the category and detection frame information of each target to be detected, the image frame is input into a multi-layer convolutional neural network with a predetermined first pixel size; Extracting a first feature representation vector of the image frame of a second pixel size through encoding operations of multiple convolutional layers and generalized normalization layers; The first pixel size is larger than the second pixel size.

4. The image compression method according to claim 1, characterized in that: The step of compressing the image frame after the distortion calculation to obtain an image of each target to be detected in the image frame compressed according to its own compression ratio includes: Determining the compression bit rate of the image frame by cross entropy; Based on the compression code rate, the image frame after the distortion calculation is compressed to obtain an image of each target to be detected in the image frame after being compressed according to its own compression ratio.

5. An image compression device based on deep learning, characterized in that: include: A detection and feature extraction module, used to obtain an image frame to be detected, detect each target to be detected in the image frame, obtain the category and detection frame information of each target to be detected, and extract features of the image frame through a multi-layer convolutional neural network according to the category and detection frame information of each target to be detected, so as to obtain a first feature representation vector of the image frame; A feature reconstruction module, used for uniformly quantizing the first feature representation vector to obtain a second feature representation vector, and reconstructing the second feature representation vector through a multi-layer convolutional neural network; A distortion calculation module, configured to perform distortion calculation on the image frame according to predefined compression ratio coefficients of various categories based on the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected; A compression module, used to compress the image frame after the distortion calculation, to obtain an image of each target to be detected in the image frame after being compressed according to its own compression ratio; The feature reconstruction module comprises: a uniform quantization unit, configured to uniformly quantize the first feature representation vector of a second pixel size to obtain a second feature representation vector of a second pixel size after uniform quantization; A training module, used for rounding quantization by adding uniform noise to obtain a uniform quantization model; The feature reconstruction module further includes: A second input unit, used to input the second feature representation vector into the multi-layer convolutional neural network; A restoration unit, configured to restore the size of the second feature representation vector of the second pixel size to the first pixel size through decoding operations of a plurality of convolutional layers and a generalized normalization layer; The distortion calculation module is specifically used to determine the regional features of different regions in each target to be detected according to the second feature representation vector after feature reconstruction, the first feature representation vector and the detection frame information of each target to be detected; Determine the category of each target to be detected, and determine the compression ratio coefficient of each target to be detected according to the predefined compression ratio coefficients of each category; The image frame is subjected to distortion calculation by a predetermined image quality evaluation function according to the regional features of each target to be detected and the compression ratio coefficient.

6. The image compression device according to claim 5, characterized in that: The detection and feature extraction module comprises: An acquisition unit, used for acquiring an image frame to be detected based on a predetermined frequency; A detection unit, configured to detect each target to be detected in the image frame by using a predetermined target detection algorithm, and obtain a detection result including a category and detection frame information of each target to be detected; The detection frame information includes pixel coordinate information of each vertex of the detection frame.

7. The image compression device according to claim 6, characterized in that: The detection and feature extraction module comprises: A first input unit, configured to input the image frame into a multi-layer convolutional neural network with a predetermined first pixel size according to the category and detection frame information of each target to be detected; An extraction unit, configured to extract a first feature representation vector of the image frame of a second pixel size through encoding operations of a plurality of convolutional layers and a generalized normalization layer; The first pixel size is larger than the second pixel size.

8. The image compression device according to claim 5, characterized in that: The compression module is specifically used to determine the compression bit rate of the image frame through cross entropy; Based on the compression code rate, the image frame after the distortion calculation is compressed to obtain an image of each target to be detected in the image frame after being compressed according to its own compression ratio.