Image feature compression and decompression methods that combine human vision and machine vision

Abstract

The application provides an image feature compression and decompression method based on human eye vision and machine vision cooperation. At the encoding end, an importance mask map with task prior information is generated and added to the local channel of an image feature extractor to obtain compact basic features. The local channel features with task prior information in the compact basic features are fused in multiple scales to obtain compact auxiliary features. Based on the compact basic features and the compact auxiliary features, a code stream for compression transmission is obtained to complete image feature compression. At the decoding end, the compact quantized basic features are reconstructed and decoded to obtain a reconstructed image for human eye vision. The compact quantized basic features and the compact quantized auxiliary features are fused to obtain compact quantized double-flow features for visual tasks. The compact quantized double-flow conversion features are obtained by aligning the compact quantized double-flow features with the features required by the visual tasks to complete task decoding. The application has more flexibility in practical application.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, specifically to an image feature compression and decompression method based on the collaboration of human vision and machine vision, and also provides a corresponding compression and decompression system, computer terminal, and computer-readable storage medium. Background Technology

[0002] In recent years, due to the unprecedented proliferation of multimedia acquisition, processing, and display devices, visual information has experienced explosive growth. To efficiently transmit and store image signals, compression techniques are typically used, leveraging characteristics such as spatial redundancy, structural redundancy, and human visual redundancy to compress the data. Traditional image coding techniques are designed for the human visual system, ensuring image quality while achieving data compression, making information loss imperceptible to the human eye. With the development of mobile IoT, numerous intelligent applications based on computer vision have emerged, such as intelligent monitoring, autonomous driving, and mobile terminals. These applications all require images as input, analyzed and processed by machine vision algorithms to complete various semantic visual tasks. Therefore, the recipient of images is no longer just the human eye, but also machines.

[0003] Generally, edge-based image acquisition systems transmit images to the cloud, where powerful computing models perform complex calculations to execute machine vision tasks. This model, known as Compression-Then-Analysis (CTA), centralizes all computations in the cloud, effectively addressing the limitation of mobile devices supporting complex calculations by deep learning models. However, big data-driven AI applications require massive amounts of data. Due to bandwidth constraints, images are typically compressed at high ratios before transmission, severely compressing image quality and impacting machine vision performance. Furthermore, the concentration of images uploaded from numerous mobile devices in the cloud can cause congestion, delaying the entire system. Therefore, this entirely cloud-based model is not optimal. Alternatively, a viable solution exists: feature extraction, compression, and transmission. This allows compact features extracted at the front end to be transmitted to the server for machine tasks—Analyze-then-Compress (ATC). Since visual features are much more compact than image textures, the performance of the analysis task is guaranteed at low bit rates. However, since the depth features extracted from the raw data cannot truly achieve signal-level image / video reconstruction, it is difficult to recover texture information.

[0004] In conclusion, while both "compress first, then analyze" and "analyze first, then compress" achieve compact representation of visual information in their own ways, neither can adequately satisfy both high-fidelity image reconstruction and high-precision visual analysis tasks at the same time.

[0005] Currently, no descriptions or reports of technologies similar to this invention have been found, and no similar information has been collected domestically or internationally. Summary of the Invention

[0006] To address the aforementioned shortcomings in the prior art, this invention provides an image feature compression and decompression method based on the collaboration of human vision and machine vision, along with a corresponding compression and decompression system, computer terminal, and computer-readable storage medium.

[0007] According to one aspect of the present invention, an image feature compression method that combines human visual perception and machine vision is provided, comprising:

[0008] At the encoding end:

[0009] A general backbone network based on machine vision is used to generate an importance mask map as prior information for the task.

[0010] After adding the importance mask image to the local channel of the image feature extractor, the image feature extractor is used to obtain compact basic features, which include local channel features with task prior information and local channel features without task prior information.

[0011] The local channel features with task prior information are fused at multiple scales to obtain compact auxiliary features;

[0012] Based on the compact basic features and the compact auxiliary features, compact quantization basic features and compact quantization auxiliary features are generated to obtain a compressed transmission bitstream, thus completing image feature compression in collaboration between human vision and machine vision.

[0013] Preferably, the machine vision-based general backbone network generates an importance mask map as prior information for the task, including:

[0014] Load the pre-trained network parameters of the general backbone network, input the original image, and average the output multi-channel task general depth features according to the channel dimension to obtain a single-channel binary two-dimensional image with the same resolution as the original image, which is the importance mask map used as the prior information of the task.

[0015] Preferably, after adding the importance mask image to the local channel of the image feature extractor, the image feature extractor is used to obtain compact basic features, including:

[0016] The importance mask image is added to some channels of the image feature extractor. The image feature extractor is then used to downsample the importance mask image by 2x, 4x and 8x respectively, and spatially fused with the channel features of different depths (low, middle and high layers) obtained by the image feature extractor.

[0017] The low-level channel features are then appended to the end of the image feature extractor for feature reconstruction supplementation.

[0018] Through the above steps, the image feature extractor performs a compact spatial transformation on the input original image through indirect bitrate constraints, obtaining local channel features with task prior information and local channel features without task prior information, thus obtaining compact basic features.

[0019] Preferably, the step of multi-scale fusion of local channel features with prior task information to obtain compact auxiliary features includes:

[0020] All compact basic features obtained by the image feature extractor are extracted respectively. The local channel features with task prior information are then fused in a multi-scale spatial manner according to the channel dimension to obtain compact auxiliary features.

[0021] Preferably, the step of generating compact quantization basic features and compact quantization auxiliary features based on the compact basic features and the compact auxiliary features to obtain the compressed transmission bitstream includes:

[0022] An entropy model is established, and the compact basic feature and the compact auxiliary feature are rounded and quantized using the entropy model to obtain the compact quantized basic feature and the compact quantized auxiliary feature, which are the compact quantized dual-stream features.

[0023] The compact quantized dual-stream features are subjected to Gaussian probability bitrate estimation using an entropy model to obtain bitrate estimates.

[0024] Entropy coding is performed on the compact quantized dual-stream features based on the bitrate estimate to obtain a compressed transmission bitstream.

[0025] According to another aspect of the present invention, an image feature decompression method combining human visual perception and machine vision is provided, comprising:

[0026] On the decoding end:

[0027] The compact quantized basic features are reconstructed and decoded to obtain a reconstructed image oriented towards human vision;

[0028] By fusing the basic compact quantization features with the auxiliary compact quantization features, compact quantization dual-stream features for vision tasks are obtained.

[0029] The compact quantized two-stream features are aligned and trained with the features required for the visual task to obtain compact quantized two-stream transformation features.

[0030] Task decoding is performed on compact quantized dual-stream conversion features to complete different analysis tasks for machine vision.

[0031] Preferably, the step of reconstructing and decoding the compact quantized basic features to obtain a reconstructed image oriented towards human vision includes:

[0032] A reconstruction decoder is constructed, which is used to convert the multi-channel compact quantization basic features into three-channel features, thereby obtaining a reconstructed image oriented towards human vision.

[0033] Preferably, the fusion of the compact quantization basic features and the compact quantization auxiliary features to obtain compact quantization two-stream features for vision tasks includes:

[0034] The compact quantization basic features and the compact quantization auxiliary features are fused according to the channel dimension to obtain compact quantization dual-stream features.

[0035] Preferably, the step of aligning and training the compact quantized two-stream features with the features required for the visual task to obtain compact quantized two-stream transformation features includes:

[0036] The compact quantized two-stream features are spatially and channel-wise upgraded to obtain features with the same size as those required for the visual task; these are called compact quantized two-stream transformation features.

[0037] Preferably, the step of performing task decoding on compact quantized dual-stream conversion features to complete different analysis tasks for machine vision includes:

[0038] Build a task decoder, including:

[0039] The pre-trained machine vision task network is divided into a head and a tail according to the location of the feature with the same size as the compact quantization two-stream conversion feature. The machine vision task network components before the location of the feature with the same size as the compact quantization two-stream conversion feature are called the head, and the machine vision task network components after the location of the feature with the same size as the compact quantization two-stream conversion feature are called the tail. The tail is used as the task decoder.

[0040] The compact quantization dual-stream conversion features are fed into the task decoder to complete different tasks for machine vision.

[0041] According to a third aspect of the present invention, an image feature compression system that combines human vision and machine vision is provided, comprising: an importance mask generation module, an image feature extractor module, a multi-scale feature fusion module, and a quantization module disposed at the encoding end; wherein:

[0042] The importance mask generation module generates an importance mask map as prior information for the task, based on a general backbone network for machine vision.

[0043] The image feature extractor module adds the importance mask image to the local channel of the image feature extractor and then uses the image feature extractor to obtain compact basic features. The compact basic features include local channel features with task prior information and local channel features without task prior information.

[0044] The multi-scale feature fusion module performs multi-scale fusion on the local channel features with task prior information to obtain compact auxiliary features.

[0045] The quantization module generates compact quantization basic features and compact quantization auxiliary features based on the compact basic features and the compact auxiliary features, obtains a compressed transmission bitstream, and completes image feature compression in collaboration between human vision and machine vision.

[0046] Preferably, VGG16 is used as the general backbone network for machine vision.

[0047] Preferably, the image feature extractor module includes: a max pooling layer, a convolutional unit, a feature fusion layer, and a long jumper unit; wherein, the max pooling layer is used to downsample the importance mask image; the convolutional unit is used to obtain channel features of different depths at low, middle, and high levels; the feature fusion layer is used to spatially fuse the downsampled importance mask image with the channel features of different depths obtained by the image feature extractor; and the long jumper unit is used to jump the low-level features to the end of the image feature extractor for feature reconstruction.

[0048] Preferably, the quantization module includes: a quantization model and an entropy model; wherein, the quantization model is used to perform rounding quantization on the compact basic feature and the compact auxiliary feature; and the entropy model performs bitrate estimation and entropy coding on the quantized dual-stream feature.

[0049] According to a fourth aspect of the present invention, an image feature decompression system that combines human vision and machine vision is provided, comprising: a reconstruction decoder module, a feature fusion module, a feature alignment module, and a task decoder module disposed at the decoding end; wherein:

[0050] The reconstruction decoder module reconstructs and decodes the compact quantization basic features to obtain a reconstructed image oriented towards human vision.

[0051] The feature fusion module fuses the compact quantization basic features and the compact quantization auxiliary features to obtain compact quantization dual-stream features for visual tasks.

[0052] The feature alignment module aligns and trains the compact quantized dual-stream features with the features required for the visual task to obtain compact quantized dual-stream conversion features.

[0053] The task decoder module performs task decoding on compact quantized dual-stream conversion features to complete different analysis tasks for machine vision.

[0054] Preferably, the reconstruction decoder includes a deconvolution unit, which performs spatial dimensionality upscaling and channel dimensionality reduction on the input compact quantized basic features to output a three-channel reconstructed image.

[0055] Preferably, the feature alignment module includes a convolutional layer and a residual block; wherein the convolutional layer is used to spatially upscale the compact quantized two-stream feature, and the residual block is used to channel upscale the compact two-stream feature, ultimately obtaining a compact quantized two-stream transformed feature with the same feature size as required by the visual task.

[0056] According to a fifth aspect of the present invention, a computer terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can be used to perform the method described in any of the preceding claims, or to run the system described in any of the preceding claims.

[0057] According to a sixth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform the methods described in any of the preceding claims, or to run the system described in any of the preceding claims.

[0058] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0059] This invention provides a method, system, computer terminal, and computer-readable storage medium for image feature compression and decompression that combines human vision and machine vision. When the image feature extractor extracts compact basic features for reconstruction tasks, it fuses prior information about the visual task with some channels of the feature extractor. This allows the features captured by the network to possess a certain semantic analysis capability while ensuring high-quality reconstructed images. At the decoding end, the reconstructed image not only achieves high-fidelity reconstruction but also better performs task analysis. Furthermore, the compact basic features can directly complete visual tasks with considerable accuracy, making this approach more flexible in practical applications.

[0060] This invention provides a method, system, computer terminal, and computer-readable storage medium for image feature compression and decompression that combines human vision and machine vision. Through the acquisition of compact basic features and compact auxiliary features, it realizes an innovative scalable dual-stream network framework. The compact basic features can accomplish high-quality image reconstruction and fairly accurate visual analysis tasks, while the compact auxiliary features extracted additionally from the feature extractor effectively supplement the compact basic features, achieving high-precision visual analysis. Compared with the most advanced traditional image compression techniques, it can effectively improve visual analysis performance under the same transmission bit requirements.

[0061] This invention provides a method, system, computer terminal, and computer-readable storage medium for image feature compression and decompression that combines human vision and machine vision. It implements a feature alignment technique. Since compact basic features for reconstruction tasks still contain information irrelevant to the visual task, this affects the accuracy of subsequent task analysis. Feature alignment uses the difference between the task target features and the compact basic features as a constraint, ensuring feature size consistency while effectively filtering out useless features and further saving coding bits.

[0062] Experiments have shown that the image feature compression and decompression method, system, computer terminal, and computer-readable storage medium provided by this invention, which combine human vision and machine vision, make compact dual-stream features more versatile and generalizable, enabling them to perform multiple visual analysis tasks simultaneously. Attached Figure Description

[0063] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0064] Figure 1 This is a flowchart illustrating the image feature compression method that combines human vision and machine vision in a preferred embodiment of the present invention.

[0065] Figure 2This is a schematic diagram of the composition structure of an image feature compression system that combines human vision and machine vision in a preferred embodiment of the present invention.

[0066] Figure 3 This is the network structure corresponding to each module in the image feature compression method that combines human visual perception and machine vision in a specific application example of the present invention.

[0067] Figure 4 The image PR curves for image reconstruction using compact basic features are shown in a specific application example of the present invention. Among them, (a) is the PSNR performance-bit rate (BPP) curve, and (b) is the SSIM performance-bit rate (BPP) curve. Bit rate (BPP) represents the number of bits required per pixel.

[0068] Figure 5 The figures show the PR curves for reconstructing images and compact two-stream conversion features to complete visual tasks in a specific application example of the present invention; where (a) is the mAP accuracy-bit rate (BPP) curve for the object detection task, and (b) is the Accuracy-bit rate (BPP) curve for the image classification task, where bit rate (BPP) represents the number of bits required per pixel.

[0069] Figure 6 This is the PR curve (mAP accuracy-bitrate curve) for completing the target detection task using compact basic features and compact dual-stream features in a specific application example of the present invention.

[0070] Figure 7 In the middle (a) to (g), the visualization results are shown. Each row, from left to right, represents the original input image, the reconstructed image, the object detection result, and the image classification result. The corresponding test values ​​are the basic feature code rate, the auxiliary feature code rate, SSIM, and PSNR, respectively. Detailed Implementation

[0071] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0072] One embodiment of this invention provides an image feature compression method that integrates human vision and machine vision. This method considers that both humans and machines can serve as the final recipients of visual data and proposes a deep learning-based image feature compression method that integrates human vision and machine vision. It can simultaneously complete high-quality image reconstruction tasks conforming to human vision and high-precision semantic analysis tasks conforming to machine vision at the decoding end. This method adds an importance mask map of the visual task to the local channels of the image feature extractor, performs compact feature representation on the original image, and obtains compact quantized basic features and compact quantized auxiliary features. The compact quantized basic features are used to complete image reconstruction for human vision. The compact quantized basic features and compact quantized auxiliary features are fused and aligned for training to obtain compact quantized two-stream conversion features, completing the analysis task for machine vision.

[0073] like Figure 1 As shown, the image feature compression method that combines human vision and machine vision provided in this embodiment may include:

[0074] At the encoding end:

[0075] S11: A general backbone network based on machine vision, which generates an importance mask map with prior task information;

[0076] S12: After adding the importance mask image to the local channel of the image feature extractor, the image feature extractor is used to obtain local channel features with task prior information and local channel features without task prior information, forming a compact basic feature.

[0077] S13: Multi-scale fusion of local channel features with task prior information to obtain compact auxiliary features;

[0078] S14: Generate compact quantization basic features and compact quantization auxiliary features based on compact basic features and compact auxiliary features, obtain compressed transmission bitstream, and complete image feature compression in collaboration between human vision and machine vision.

[0079] The embodiments of this invention combine prior information from visual tasks with an image feature extractor to extract compact basic features and compact auxiliary features. The compact basic features perform image reconstruction and semantic analysis tasks with considerable accuracy. Then, the compact basic features and compact auxiliary features are fused and aligned during training to obtain compact dual-stream aligned features, which complete the high-precision semantic analysis task. Compared with state-of-the-art traditional image compression methods, this invention exhibits better performance in both human and machine vision.

[0080] In a preferred embodiment of S11, the pre-trained network parameters of the general backbone network are loaded, the original image is input, and the output multi-channel task-specific depth features are averaged according to the channel dimension to obtain a single-channel binary two-dimensional image with the same resolution as the original image, which serves as the importance mask map for task prior information. Further,

[0081] VGG16 was chosen as the general backbone network for visual tasks. The original image was fed into the pre-trained VGG16 to obtain the last layer feature map before the classifier. The multi-channel feature map was then aggregated, resized, and binarized according to the channel dimensions to obtain a single-channel binary importance mask map of the same size as the original image.

[0082] In a preferred embodiment of S12, an importance mask image is added to some channels of the image feature extractor. The image feature extractor then downsamples the importance mask image by 2x, 4x, and 8x, respectively, and spatially fuses it with the low-level, mid-level, and high-level channel features obtained by the image feature extractor. The low-level channel features are then appended to the end of the image feature extractor (i.e., the end of the fused features) for feature reconstruction. Through these steps, the image feature extractor performs a compact spatial transformation on the input original image through indirect bitrate constraints, obtaining local channel features with task prior information and local channel features without task prior information, thus obtaining compact basic features. Further,

[0083] The importance mask image obtained by S11 is downsampled by 2, 4 and 8 times respectively, and then spatially fused with the first 1 / 4 channel features of low, medium and high depths of the feature extractor. Next, the shallow features of the image feature extractor without prior information are jumped to the end of the image feature extractor to reconstruct and supplement the features, and finally the compact basic features are obtained.

[0084] In a preferred embodiment of S13, all compact basic features obtained by the image feature extractor are extracted respectively, and the local channel features with task prior information are spatially fused at multiple scales according to the channel dimension to obtain compact auxiliary features. Further,

[0085] The image feature extractor extracts low-level, mid-level, and high-level features, which are three-dimensional features with feature sizes of C*256*256, C*128*128, and C*64*64, respectively, where C refers to the number of channels in the feature. The first quarter of the features, which are added to the prior, are fused sequentially according to channel dimension. Considering the scale inconsistency between features of different depths, shallow features are first pooled to achieve size matching with deeper features before channel dimension superposition, ultimately resulting in compact auxiliary features. The specific fusion method is as follows: the importance mask is downsampled by 2 and multiplied with a portion of the shallow features, then the original shallow features are added; the importance mask is downsampled by 4 and multiplied with a portion of the mid-level features, then the original mid-level features are added; the importance mask is downsampled by 8 and multiplied with a portion of the deep features, then the original deep features are added.

[0086] In a preferred embodiment of S14, an entropy model is established, and the compact basic features and compact auxiliary features are rounded and quantized using the entropy model to obtain compact quantized basic features and compact quantized auxiliary features, which are the compact quantized dual-stream features; the compact quantized dual-stream features are then subjected to Gaussian probability bitrate estimation using the entropy model to obtain bitrate estimates; based on the bitrate estimates, the compact quantized dual-stream features are then entropy encoded to obtain the compressed transmission bitstream. Further, S14 specifically includes:

[0087] The compact basic features obtained in S13 and the compact auxiliary features obtained in S14 are rounded and quantized. The feature value distribution is modeled by Gaussian probability using an entropy model to achieve end-to-end rate distortion optimization, and finally the bit rate estimate and binary bit stream are obtained.

[0088] In the above embodiments, the compact quantization basic features are used as features of human visual perception; the compact quantization basic features and the compact quantization auxiliary features are used as features of machine vision.

[0089] One embodiment of the present invention provides an image feature decompression method that combines human vision and machine vision.

[0090] like Figure 1 As shown, the image feature decompression method that combines human vision and machine vision in this embodiment may include:

[0091] On the decoding end:

[0092] S15: Reconstruct and decode the compact quantized basic features to obtain a reconstructed image oriented towards human vision;

[0093] S16: Fuse the compact quantization basic features and compact quantization auxiliary features to obtain compact quantization two-stream features for vision tasks;

[0094] S17: Align and train the compact quantized two-stream features with the features required for the visual task to obtain compact quantized two-stream transformation features;

[0095] S18: Perform task decoding on compact quantized dual-stream conversion features to complete different analysis tasks for machine vision.

[0096] In a preferred embodiment of S15, a reconstruction decoder is constructed to convert the multi-channel compact quantized basic features into three-channel features, thereby obtaining a reconstructed image oriented towards human vision. Further, S15 specifically includes:

[0097] By performing the inverse process of a class feature extractor on the multi-channel compact quantization basic features, channel dimensionality reduction and spatial dimensionality enhancement are achieved to restore the original feature size, thereby completing image reconstruction.

[0098] In a preferred embodiment of S16, the compact quantization basic features and compact quantization auxiliary features are fused along the channel dimension to obtain compact quantization dual-stream features. Further, S16 specifically includes:

[0099] First, the basic and auxiliary features of compact quantization are batch normalized. Second, the two-stream features are fused according to the channel dimension to obtain compact quantization dual-stream features.

[0100] In a preferred embodiment of S17, the compact quantized two-stream features are spatially and channel-wise upgraded to obtain compact quantized two-stream transformation features with the same feature size as required by the visual task.

[0101] In a preferred embodiment of S18,

[0102] The pre-trained machine vision task network is divided into a head and a tail based on the location of features with the same size as those aligned with feature S17. The first part of the network serves as the head, and the second part as the tail. The tail then functions as the task decoder. The compact quantized dual-stream transformed features obtained from S17 are fed into the task decoder to decompress image features for different machine vision tasks.

[0103] One embodiment of the present invention provides an image feature compression system that combines human vision and machine vision.

[0104] like Figure 2 As shown, the image feature compression system for the collaboration of human vision and machine vision provided in this embodiment may include: an importance mask generation module, an image feature extractor module, a multi-scale feature fusion module, and a quantization module, all located at the encoding end; wherein:

[0105] The importance mask generation module, based on a general backbone network for machine vision, generates an importance mask map with prior task information.

[0106] The image feature extractor module adds the importance mask image to the local channel of the image feature extractor and then uses the image feature extractor to obtain compact basic features. The compact basic features include local channel features with task prior information and local channel features without task prior information.

[0107] The multi-scale feature fusion module fuses local channel features with prior task information at multiple scales to obtain compact auxiliary features.

[0108] The quantization module generates compact quantization basic features and compact quantization auxiliary features based on compact basic features and compact auxiliary features, obtains a compressed transmission bitstream, and completes image feature compression in collaboration between human vision and machine vision.

[0109] In a preferred embodiment, vgg16 is used as the general backbone network for machine vision.

[0110] In a preferred embodiment, the image feature extractor module includes: a max pooling layer, a convolutional unit, a feature fusion layer, and a long jumper unit; wherein, the max pooling layer is used to downsample the importance mask image; the convolutional unit is used to obtain channel features of different depths at low, middle, and high levels; the feature fusion layer is used to spatially fuse the downsampled importance mask image with the channel features of different depths obtained by the image feature extractor; and the long jumper unit is used to jump low-level features to the end of the image feature extractor for feature reconstruction.

[0111] In a preferred embodiment, the quantization module includes: a quantization model and an entropy model; wherein the quantization model is used to perform rounding quantization on compact basic features and compact auxiliary features; and the entropy model performs bitrate estimation and entropy coding on the quantized features.

[0112] An embodiment of the present invention also provides an image feature decompression system that combines human vision and machine vision.

[0113] like Figure 2 As shown, the image feature decompression system for human vision and machine vision collaboration provided in this embodiment may include: a reconstruction decoder module, a feature fusion module, a feature alignment module, and a task decoder module disposed at the decoding end; wherein:

[0114] The reconstruction decoder module reconstructs and decodes the compact quantized basic features to obtain a reconstructed image oriented towards human vision.

[0115] The feature fusion module fuses the compact quantization basic features and compact quantization auxiliary features to obtain compact quantization dual-stream features for visual tasks.

[0116] The feature alignment module aligns and trains compact quantized two-stream features with the features required for the visual task to obtain compact quantized two-stream transformation features.

[0117] The task decoder module performs task decoding on compact quantized dual-stream conversion features to complete different analysis tasks for machine vision.

[0118] In a preferred embodiment, the reconstruction decoder includes a deconvolution unit for spatial dimensionality upscaling and channel dimensionality downscaling of the input compact quantized basic features, outputting a three-channel reconstructed image.

[0119] In a preferred embodiment, the feature alignment module includes a convolutional layer and a residual block; wherein the convolutional layer is used to spatially upscale the compact quantized two-stream features, and the residual block is used to channel upscale the compact two-stream features, ultimately obtaining compact quantized two-stream transformed features with the same feature size as required by the visual task.

[0120] In a preferred embodiment, the task decoder module is used to construct the task decoder, wherein the pre-trained machine vision task network is divided into a head and a tail according to the location of the features that are consistent with the size of the compact quantized two-stream transform features, wherein the first part of the network is the head and the second part is the tail, and the tail is used as the task decoder; the compact quantized two-stream transform features are fed into the task decoder to complete different analysis tasks for machine vision.

[0121] The compression and decompression methods and systems provided in the above embodiments of the present invention will be further described in detail below with reference to a specific application example.

[0122] This specific application example demonstrates the implementation of the above method through programming simulation in a Win10 system and PyTorch environment. Figure 2 This is a schematic diagram illustrating the structural composition of a deep learning-based image feature compression method that integrates human and machine vision in a specific application example of the present invention. Figure 3 This is the network structure corresponding to each module in the image feature compression method that combines human visual perception and machine vision in a specific application example of the present invention.

[0123] The compression and decompression method and system include:

[0124] At the encoding end:

[0125] S11: A general backbone network based on machine vision generates an importance mask map with prior task information.

[0126] First, an importance mask generation module is established. VGG16 is selected as the general backbone network for the task. The original image X is fed into the pre-trained VGG16 to obtain the last feature map before the classification layer of VGG16. A series of transformation operations such as channel dimension aggregation, size transformation and binarization are performed on the feature map to obtain a single-channel binary importance mask map with the same size as the original image.

[0127] Mask = H(vgg16) feature (X)) (1)

[0128] In equation (1), Mask is a binary importance mask image, and H(·) is a series of transformation operations;

[0129] In S11, the last layer of feature map is the basis for generating a single-channel binary importance mask map. The importance mask map is obtained by performing a series of transformation operations on the feature map, such as channel dimension aggregation, size transformation and binarization.

[0130] S12: After adding the importance mask image to the local channel of the image feature extractor, the image feature extractor is used to obtain compact basic features with task prior information:

[0131] A priori local fusion image feature extractor is established, consisting of a max pooling layer, a convolutional unit, a feature fusion layer, and a long jumper unit. The max pooling layer downsamples the importance mask image by 2, 4, and 8 times, as shown in the following formula.

[0132]

[0133] In equation (2), Mask 2x To downsample the importance mask image by 2 times, Mask 4x To downsample the importance mask by 4 times, Mask 8x The importance mask is downsampled by 8 times, and Maxpool(·) is the max pooling layer;

[0134] The convolutional unit downsamples the original image to achieve compact feature extraction, and the feature fusion layer fuses the importance mask image with the first 1 / 4 channels of low, medium and high depth features respectively.

[0135]

[0136] F′ k [:,0:a,:,:]=F k [:,0:a,:,:]·Mask i +F k [:,0:a,:,:],k∈{low,mid,high},i∈{2x,4x,8x} (4)

[0137] In equation (3), in this specific embodiment, a is 48 and C is 192;

[0138] In equation (4), importance mask fusion is performed layer by layer. Therefore, taking the first layer fusion process as an example, F′ k [:,0:a,:,:] represents the first 48 channels of the low-level features obtained after importance mask fusion. i For the importance mask image with 2x downsampling, F k [:,0:a,:,:] represents the first 48 channels of the original low-level features; the same applies to the mid-level and high-level features.

[0139] The long jump unit jumps low-level features that have not been incorporated with prior information to the end of the image feature extractor to supplement the reconstructed features, and finally obtains the compact basic features of the image, as shown in the following formula.

[0140] F base =F′ high +F low (5)

[0141] In equation (5), F base For compact basic features of the image, F′ high For the high-level characteristics after integration, F low These are low-level features that have not incorporated prior information.

[0142] S13: Multi-scale fusion of local channel features incorporating task prior information from the compact basic features yields compact auxiliary features:

[0143] A multi-scale feature fusion module is established. The low, medium and high depth features obtained by S12 are input, and the first 1 / 4 channel features with prior are fused sequentially according to the channel dimension. Considering the scale inconsistency between features of different depths, the shallower features need to be downsampled first to achieve size matching with the deeper features, and then the channel dimensions are superimposed to finally obtain compact auxiliary features, as shown in the following formula.

[0144] F fusion1 =Cat(BN(conv(F′) low ),BN(conv(F′ mid (6)

[0145] F extra =Cat(BN(conv(F) fusion1 ),BN(conv(F′ high ))

[0146] In equation (6), F fusion1 F represents the characteristics of the fused low- and mid-level layers.extra For compact auxiliary features, Cat(·) is used for channel stacking and fusion, BN(·) is used for batch normalization, conv(·) is used for convolutional layers, and F′ is used for... low F′ mud F′ high These are the low, medium, and high-level features obtained after fusing important new masks;

[0147] S14: Using the entropy model, quantization, bitrate estimation, and entropy coding are performed on the compact basic features and compact auxiliary features respectively to obtain the compact quantized basic features and compact quantized auxiliary features:

[0148] Establish an entropy model and input compact basic features. First, use rounding quantization to map continuous feature values ​​to discrete values, as shown in the following formula.

[0149]

[0150] In equation (7), For compact quantization, the basic feature is rounding (·), which is rounding quantization.

[0151] Considering that rounding quantization is not differentiable in deep learning backpropagation, uniform noise is used instead of rounding during the training phase, as shown in the following formula.

[0152]

[0153] In equation (8), noise(-0.5,+0.5) is uniform noise with a mean of 0 and a range of [-0.5,+0.5].

[0154] An entropy model is used to perform Gaussian modeling on the distribution of compact basic feature values ​​to obtain a bit rate estimate, and then entropy coding is performed to obtain a binary bit stream.

[0155] On the decoding end:

[0156] S15: Feed the compact quantized basic features into the reconstruction decoder to obtain a reconstructed image oriented towards human vision:

[0157] A reconstruction decoder is established, whose network structure is roughly symmetrical to the image feature extractor. It inputs multi-channel compact quantized basic features to achieve feature channel dimensionality reduction and spatial dimensionality enhancement, thereby completing image reconstruction, as shown in the following formula, where D is the reconstruction function.

[0158]

[0159] In equation (9), Re represents the reconstructed image;

[0160] S16: Fuse the compact quantization basic features and compact quantization auxiliary features to obtain compact quantization two-stream features for vision tasks:

[0161] Batch normalization is performed on the basic features and auxiliary features of compact quantization respectively. Then, the fusion operation is performed according to the channel dimension, as shown in the following formula.

[0162]

[0163] In equation (10), To compactly quantize the two-stream characteristics, For compact quantization auxiliary features;

[0164] S17: Align and train the compact quantized two-stream features with the features required for the visual task to obtain compact quantized two-stream transformation features:

[0165] A feature alignment module is established, consisting of convolutional layers and residual blocks. The convolutional layers perform spatial dimensionality upscaling on the compact quantized two-stream features, and the residual blocks perform channel dimensionality upscaling on the compact two-stream features, resulting in compact quantized two-stream transformed features with the same size as the task features, as shown in the following formula, where T represents the transformation function.

[0166]

[0167] In equation (11), To compactly quantize the characteristics of the two-stream conversion, F task Features required for the task;

[0168] S18: Perform task decoding on compact quantized dual-stream conversion features to complete different machine vision tasks:

[0169] Establishing a task decoder includes: dividing the pre-trained machine vision task network into a head and a tail according to the location of features with the same size as the compact quantized two-stream transform features, wherein the first part of the network is the head and the second part is the tail, and the tail is used as the task decoder; feeding the compact quantized two-stream transform features into the task decoder to complete different analysis tasks for machine vision.

[0170] Before performing network inference, the modules corresponding to the above steps need to be trained. First, the reconstruction branches (i.e., branches that can independently complete human visual tasks, such as...) are trained. Figure 2 As shown, it includes an importance mask generation module, an image feature extractor module, a part of the quantization module used for compact basic feature quantization (quantization module 1), and a reconstruction decoder module. Its loss function is shown in the following formula: R base For compact quantization of basic feature bitrate:

[0171] L rec =MSE(X,Re)+α·(1-SSIM(X,Re)) (12)

[0172] Loss1=λ1·L rec +R base (13)

[0173] In equation (12), L rec For reconstruction loss, | is the input image, α is the Lagrange factor, and MSE(·) and SSIM(·) are the image distortion measures;

[0174] In equation (13), Loss1 is λ1, and λ1 is the Lagrange factor;

[0175] Secondly, fix the reconstruction branch and the training task branch (i.e., the branch that can complete the visual task, such as...). Figure 2 As shown, it includes: a feature fusion module, a part of the quantization module used for compact auxiliary feature quantization (quantization module 2), a feature alignment module, and a task decoder module. Its loss function is shown in the following formula: R extra To compactly quantize the auxiliary feature code rate:

[0176]

[0177] Loss2=λ2·L fea +(R base +R extra (15)

[0178] In equation (14), L fea For feature loss;

[0179] In equation (15), Loss2 is λ2, and λ2 is the Lagrange factor.

[0180] Finally, the entire network is jointly fine-tuned end-to-end, and the loss function (Loss) is shown in the following formula:

[0181] Loss=λ1·L rec +λ2·L fea +(R base +R extra (16)

[0182] In this specific application example, the effectiveness of the feature compression method provided in the above embodiments of the present invention is evaluated through experiments on public datasets. Specifically, image reconstruction performance is evaluated on the Kodak24 dataset, object detection performance on the PASCALVOC dataset, and image classification performance on the mini-imagent dataset. The experimental environment is a Win10 system and the PyTorch framework. The evaluation metrics are the performance of multiple tasks under different bitrate conditions. For example, the performance of image reconstruction tasks based on human vision is evaluated using image quality metrics PSNR and SSIM, while the performance of analysis tasks based on machine vision is evaluated using mean accuracy (mAP) or accuracy. The image object detection visual task can use the SSD network, and the image classification visual task can use the ResNet network. The performance-rate (PR) curve represents the compression performance; the further to the upper left of the curve, the better the compression performance. In this experiment, the visual tasks are image object detection and image classification.

[0183] First, for image reconstruction tasks targeting human vision, compared to the most advanced traditional image compression techniques currently available, such as... Figure 4 As shown in (a) and (b), at the same compression bitrate, the present invention has superior performance in PSNR and outperforms other existing methods in SSIM.

[0184] Secondly, for machine vision-oriented object detection tasks, compared with the most advanced traditional image compression technologies, such as... Figure 5 As shown in (a), the performance curve corresponding to the present invention is always at the top, which means that at the same bit rate, whether the reconstructed image is sent into the task network or the semantic task is completed directly using compact quantized dual-stream conversion features, the average precision mAP of the present invention is higher, indicating that the target detection performance is better.

[0185] To verify the generalization and universality of the dual-stream features, compact quantization basic features, and compact quantization auxiliary features transmitted in this invention, dual-stream features for the target detection task transmitted by the image feature extractor were extracted and fed into a classification network. Experimental results are as follows: Figure 5 As shown in (b), the performance curve corresponding to the present invention is always at the top, that is, under the same bitrate, the performance of the present invention is also the best in classification tasks, indicating that the extracted dual-stream features still perform well in other tasks, further illustrating that the features proposed by the present invention have universality and generalization.

[0186] Secondly, ablation experiments were conducted to verify the performance improvement effect of compact quantization auxiliary features on the final machine vision task. Two sets of compressed models were trained. The first set of compressed models was the two-stream network model proposed in this invention, and the second set of compressed models was a single-stream network model without the multi-scale feature fusion module. Specifically, the single-stream network model directly fed the compact quantization basic features into the feature alignment module. The machine vision task selected was object detection, and the experimental results are as follows: Figure 6 As shown, with a bitrate in the range of 0.18 to 0.58, the accuracy using compact basic features can be maintained above 59%, while the accuracy using dual-stream features can be maintained above 70%. Furthermore, according to the PR curve, the curve corresponding to the dual-stream model with the addition of compact quantization auxiliary features is always on the top, verifying the performance improvement effect of compact quantization auxiliary features on the task.

[0187] Figure 7 Images (a)-(g) show the visualization results of different tasks performed by the compressed and decompressed features obtained through the technical solutions provided in the above embodiments of the present invention. The numerical values ​​represent the bit rate (BPP) after basic feature compression, the bit rate (BPP) after auxiliary feature compression, and the reconstructed image quality metrics SSIM and PSNR (dB), respectively. The results show that image feature compression based on the technical solutions provided in the above embodiments of the present invention not only reconstructs images with good visual quality but also effectively completes object detection and image classification tasks.

[0188] The task decoder can directly perform various visual tasks using input features. Taking the object detection visual task as a specific application example, the task network uses an SSD object detection network. The task decoder is the tail network part (also called the downstream network part) of the SSD object detection network. Specifically, it feeds the compact quantized two-stream transform features into the task decoder, and the output is the object detection result, i.e., the category and location of the object in the input image. Experimental results are reflected in: quantitative results such as... Figure 5 As shown in (a), this reflects the detection accuracy. Visually, the experimental results can be understood. Figure 7 The third column contains the location of the calibration box and the classification probability.

[0189] For object detection tasks, the task decoder takes "compactly quantized two-stream transform features" as input and outputs the object detection results, i.e., the category and location of objects in the image. The results are reflected in the detection accuracy (mAP) and the visual bounding boxes (e.g., ...). Figure 7 (Column 3); For image classification tasks, the input to the task decoder is "compact quantized two-stream transform features," and the output is the image classification result, i.e., the category of objects in the image. The result is reflected in classification accuracy and visual labels (e.g., ...). Figure 7 (Column 4 in the middle).

[0190] As can be seen from the above experiments, the technical solution provided by the above embodiments of the present invention can realize collaborative image feature compression for human vision and machine vision, improve image reconstruction quality, and enhance target detection and image classification performance.

[0191] It should be noted that the steps in the method provided by the present invention can be implemented using corresponding modules, devices, units, etc. Those skilled in the art can refer to the technical solution of the method to realize the composition of a specific system. That is, the embodiments in the method can be understood as preferred examples for building the system, and will not be elaborated here.

[0192] One embodiment of the present invention provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform any of the methods in the above embodiments of the present invention, or to run any of the systems in the above embodiments of the present invention.

[0193] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0194] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0195] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0196] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0197] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform the method of any of the above embodiments of the present invention, or to run the system of any of the above embodiments of the present invention.

[0198] The image feature compression and decompression method, system, computer terminal, and computer-readable storage medium for collaborative human and machine vision provided in the above embodiments of the present invention combine prior information of visual tasks with traditional depth image compression coding techniques to realize an innovative two-stream framework for collaborative human and machine vision image feature compression and decompression. Under this two-stream framework, if only compact basic features are transmitted to the decoding end, the following functions can be performed at the decoding end: a high-quality reconstructed image meeting human visual requirements (e.g., the image reconstruction task in the specific embodiment, the result of which is...) can be obtained through the reconstruction decoder module. Figure 4 The proposed curve and Figure 7 The second column of the image is reconstructed; the feature alignment module and the task decoder module complete the analysis task with a high degree of accuracy in line with machine vision (such as the object detection task in the specific embodiment, the result is...). Figure 6 Compact basic feature curve); If, in addition to transmitting compact basic features, compact auxiliary features are also transmitted, then at the decoding end, these two streams of features can be fused to obtain compact quantized dual-stream features. This allows the following functions to be performed at the decoding end: High-precision analysis tasks conforming to machine vision (such as the target detection task in a specific embodiment, with the result being...) can be completed through the feature alignment module and the task decoder module. Figure 5 (a) and Figure 6 The proposed curve and Figure 7 (The third column of results). Compared with the most advanced traditional image compression and decompression technologies, the technical solution provided by the above embodiments of the present invention can achieve higher quality image reconstruction tasks and higher precision machine vision tasks.

[0199] Those skilled in the art will understand that, in addition to implementing the embodiments and devices provided by the present invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the embodiments and devices provided by the present invention function as logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the methods and devices provided by the present invention can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0200] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method of image feature compression in collaboration with human vision and machine vision of a human eye, characterized by, Comprise: At the encoding end: Load the pre-training network parameters of the general backbone network of machine vision, input the original image, average the output multi-channel task general deep features according to the channel dimension to obtain a single-channel binary two-dimensional image with the same resolution as the original image, that is, an importance mask graph as task prior information; After adding the importance mask graph to the local channel of the image feature extractor, the compact base feature is obtained by using the image feature extractor, which includes local channel features with task prior information and local channel features without task prior information; The local channel features with task prior information are multi-scale fused to obtain a compact auxiliary feature; An entropy model is established, and the compact base feature and the compact auxiliary feature are quantized by using the entropy model to obtain a compact quantized double-flow feature; the compact quantized double-flow feature is estimated by a Gaussian probability code rate estimation model to obtain a code rate estimation value; the compact quantized double-flow feature is entropy encoded based on the code rate estimation value to obtain a compressed transmission code stream, and the image feature compression of human visual and machine visual collaboration is completed.

2. The method of image feature compression for human and machine vision synergy of the eye of claim 1, wherein, Also include any one or any multiple of the following: After adding the importance mask graph to the local channel of the image feature extractor, the compact base feature is obtained by using the image feature extractor, which includes: The importance mask graph is added to part of the channel of the image feature extractor, and the importance mask graph is down-sampled by 2, 4 and 8 times respectively by using the image feature extractor, and is fused with the channel features of different depths obtained by the image feature extractor at low, middle and high layers in spatial dimension; The channel features of the low layer are connected to the tail end of the image feature extractor for feature reconstruction; Through the above steps, the image feature extractor performs compact spatial transformation on the input original image through indirect code rate constraint to obtain local channel features with task prior information and local channel features without task prior information, and obtains a compact base feature; The local channel features with task prior information are multi-scale fused to obtain a compact auxiliary feature. Comprise:

3. An image feature decompression method for human eye vision and machine vision collaboration, characterized in that, At the decoding end: Reconstruct and decode the compact quantized base feature to obtain a reconstructed image for human visual; Fuse the compact quantized base feature and the compact quantized auxiliary feature to obtain a compact quantized double-flow feature for visual tasks; Align the compact quantized double-flow feature and the feature required by the visual task, and perform spatial dimension and channel dimension upgrade on the compact quantized double-flow feature to obtain a compact quantized double-flow conversion feature with the same size as the feature required by the visual task; Task decode the compact quantized double-flow conversion feature to complete different analysis tasks for machine vision. Also include any one or any multiple of the following:

4. The method of image feature decompression for human and machine vision synergy of the eye of claim 3, wherein, ​ - the reconstructing decoding of the compact quantized base feature to obtain a reconstructed image for human visual perception, comprising: constructing a reconstructing decoder, and converting the multi-channel compact quantized base feature into a three-channel feature by using the reconstructing decoder, so as to obtain a reconstructed image for human visual perception; - the fusing of the compact quantized base feature and the compact quantized auxiliary feature to obtain a compact quantized double-flow feature for visual tasks, comprising: fusing the compact quantized base feature and the compact quantized auxiliary feature according to the channel dimension to obtain a compact quantized double-flow feature; - the task decoding of the compact quantized double-flow converted feature to complete different analysis tasks for machine vision, comprising: constructing a task decoder, comprising: dividing a pre-trained machine vision task network into a head and a tail according to a feature position consistent with the size of the compact quantized double-flow converted feature, wherein a component of the machine vision task network before the feature position consistent with the size of the compact quantized double-flow converted feature is taken as the head, a component of the machine vision task network after the feature position consistent with the size of the compact quantized double-flow converted feature is taken as the tail, and the tail is taken as the task decoder; sending the compact quantized double-flow converted feature into the task decoder to complete different tasks for machine vision.

5. An image feature compression system for human eye vision and machine vision collaboration, characterized in that, comprising: an importance mask generation module, an image feature extractor module, a multi-scale feature fusion module and a quantization module arranged at an encoding end; wherein: the importance mask generation module loads pre-training network parameters of a general backbone network of machine vision, inputs an original image, and performs average operation on output multi-channel task general deep features according to the channel dimension to obtain a single-channel binary two-dimensional image with the same resolution as the original image, i.e. an importance mask graph serving as task prior information; the image feature extractor module adds the importance mask graph to a local channel of the image feature extractor, and then uses the image feature extractor to obtain a compact base feature, the compact base feature including a local channel feature with task prior information and a local channel feature without task prior information; the multi-scale feature fusion module performs multi-scale fusion on the local channel feature with task prior information to obtain a compact auxiliary feature; the quantization module establishes an entropy model, uses the entropy model to quantize the compact base feature and the compact auxiliary feature respectively to obtain a compact quantized base feature and a compact quantized auxiliary feature, i.e. a compact quantized double-flow feature; performs Gaussian probability code rate estimation on the compact quantized double-flow feature through the entropy model to obtain a code rate estimation value; and performs entropy encoding on the compact quantized double-flow feature based on the code rate estimation value to obtain a compressed transmission code stream, thereby completing image feature compression for human visual perception and machine visual perception.

6. The image feature compression system for human and machine vision synergy of claim 5, wherein, further comprising any one or any multiple of the following: - using vgg16 as the general backbone network of machine vision; The image feature extractor module comprises a max pooling layer, a convolution unit, a feature fusion layer and a long skip unit; the max pooling layer is configured to down-sample the importance mask map; the convolution unit is configured to obtain channel features at different depths of low, middle and high layers; the feature fusion layer is configured to perform spatial dimension fusion of the down-sampled importance mask map and channel features at different depths obtained by the image feature extractor; and the long skip unit is configured to skip low layer features to the tail end of the image feature extractor for reconstruction feature supplement. The quantization module comprises a quantization model and an entropy model; the quantization model is configured to round quantize the compact base feature and the compact auxiliary feature; and the entropy model is configured to perform code rate estimation and entropy coding on the quantized double-flow feature.

7. An image feature decompression system for human eye vision and machine vision collaboration, characterized in that, The method comprises: The reconstruction decoder module, the feature fusion module, the feature alignment module and the task decoder module are arranged at the decoding end; wherein: The reconstruction decoder module performs reconstruction decoding on the compact quantized base feature to obtain a reconstruction image facing human eye vision; The feature fusion module fuses the compact quantized base feature and the compact quantized auxiliary feature to obtain compact quantized double-flow features facing visual tasks; The feature alignment module performs alignment training on the compact quantized double-flow features and features required by visual tasks, and performs spatial dimension increase and channel dimension increase on the compact quantized double-flow features to obtain compact quantized double-flow conversion features with the same size as the features required by visual tasks; The task decoder module performs task decoding on the compact quantized double-flow conversion features to complete different analysis tasks facing machine vision.

8. The image feature decompression system for human and machine vision synergy of the eye of claim 7, wherein, Any one or any multiple of the following is further included: The reconstruction decoder comprises a deconvolution unit configured to perform spatial dimension increase and channel dimension decrease on the input compact quantized base feature to output a three-channel reconstruction image; The feature alignment module comprises a convolution layer and a residual block; the convolution layer is configured to perform spatial dimension increase on the compact quantized double-flow features, and the residual block is configured to perform channel dimension increase on the compact quantized double-flow features to finally obtain compact quantized double-flow conversion features with the same size as the features required by visual tasks.

9. A computer terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to perform the method of any one of claims 1-4.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to perform the method of any one of claims 1-4. The program is executed by the processor to perform the method of any one of claims 1-4.

Images