A cross-scale feature compression method for machine vision tasks

Abstract

This invention belongs to the field of image compression and computer vision technology, and discloses a cross-scale feature compression method for machine vision tasks. The method first extracts multi-scale features using a pre-trained feature pyramid network, then compresses them into two single-scale features through a hierarchical feature fusion network. Finally, a cross-scale entropy coding network combines a super-prior and a context model for joint probability modeling and entropy coding. A hierarchical feature reconstruction network then reconstructs the multi-scale feature pyramid. During the training phase, a rate-distortion joint optimization method is employed, simultaneously constraining the bit rate, feature reconstruction error, and detection / segmentation task loss, enabling the compression process to adaptively serve the machine vision task. Compared with existing image and feature compression methods, this invention provides directly usable features for downstream tasks without requiring image reconstruction or repeated feature extraction, significantly reducing bit rate and decoding computational complexity while maintaining or improving task accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image compression and computer vision technology, and specifically relates to a cross-scale feature compression method for machine vision tasks. Background Technology

[0002] With the widespread application of artificial intelligence and the Internet of Things, image and video data are increasingly being analyzed and processed by machines rather than being directly viewed by the human eye. Therefore, Image Coding for Machines (ICM) has become an important research direction. Unlike traditional image codecs (such as JPEG, H.265 / HEVC, and H.266 / VVC) that aim to optimize the subjective visual experience of humans, the core objective of ICM technology is to efficiently compress image data while preserving as much key information as possible that is useful for downstream machine vision tasks (such as object detection and instance segmentation).

[0003] ICM's technical approach is mainly divided into two categories: image compression and feature compression. Image compression still follows the "encode-decode-analyze" process, but during encoding, it employs preprocessing, post-processing, or bitrate-distortion optimization strategies to make the compressed image more suitable for machine analysis. However, this approach still requires feature extraction from the reconstructed image at the decoding end, introducing additional computational overhead, and its overall compression efficiency is limited by the traditional coding framework.

[0004] To address these issues, deep feature compression technology has emerged. This approach places the feature extraction step before the encoder, directly compressing and transmitting the deep features extracted from the neural network. The decoder, upon receiving the compressed features, can directly use them for subsequent visual tasks without reconstructing the image or repeatedly extracting features, significantly reducing the decoder's computational complexity. However, deep features inherently suffer from high dimensionality and large data volume. Especially in modern machine vision networks, the Feature Pyramid Network (FPN) generates multi-scale features, with a total data volume far exceeding that of the original input image, posing a significant challenge to efficient compression.

[0005] Existing feature compression methods typically reduce data volume by decreasing the spatial resolution or number of channels of the feature map, or by rearranging feature channels to utilize the correlation between channels. Some end-to-end learning methods attempt to fuse multi-scale features into a compact representation at a single scale before compression. This leads to difficulties in recovering the original multi-scale feature information with high quality during decoding, and existing methods generally ignore the correlation between features at different scales, causing data redundancy. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a cross-scale feature compression method for machine vision tasks. This method designs an end-to-end compression framework that, while ensuring the performance of downstream tasks, deeply mines and eliminates redundancy between multi-scale features, thereby achieving a high compression ratio while preserving as much key information valuable to subsequent machine vision tasks as possible.

[0007] The technical solution adopted in this invention is as follows:

[0008] A cross-scale feature compression method for machine vision tasks includes the following steps:

[0009] Step 1: Multi-scale feature extraction;

[0010] Input image The input is fed into a pre-trained Feature Pyramid Network (FPN) to extract multi-scale features containing multiple scales. , , Pyramid features represent feature map information at different scales. Each scale of feature map has the same number of channels but different spatial resolutions.

[0011] Step 2: Network setup;

[0012] The cross-scale feature compression network is constructed, consisting of three parts: the first part is a hierarchical feature fusion (HFF) network; the second part is a cross-scale entropy coding (CSEC) network; and the third part is a hierarchical feature reconstruction (HFR) network. The process of constructing the cross-scale feature compression network to process multi-scale features is as follows:

[0013] Step 21: Layered feature fusion;

[0014] The HFF network is used to process the multi-scale features extracted in step 1. Based on their importance to the machine vision task, features are grouped and fused into two scales. and The specific implementation is as follows:

[0015] The HFF network consists of network units. , , and The components are connected sequentially, specifically as follows:

[0016] Network Unit The residual downsampling block has the following structure: ResDown + ResBlock. ResDown consists of a backbone and a bypass structure. The backbone is Conv 3×3 + Norm + ReLU + Conv 3×3, and the bypass is Conv 1×1. The input of Conv 3×3 in the backbone passes through the bypass Conv 1×1 and is then fused with the output of the backbone to obtain the output of ResDown. The main structure of ResBlock is Conv 3×3 + ReLU + Conv 3×3 + ReLU. The output of the main structure is added to the input to obtain the ResBlock output. (Network unit) Input Features of pyramids The output is ;

[0017] Network Unit The structure is: Concat( The network unit consists of ResDown, ResBlock, and Attention modules, where the Attention module is the attention module itself. Input for The output is , Indicates by , Fusion of low-level single-scale features;

[0018] Network Unit The structure is: ResDown + ResBlock; network unit Input for The output is ;

[0019] Network Unit The structure is: Concat( ) + Conv 3×3 + Attention; Network Unit Input for The output is , Indicates by , The fusion of high-level single-scale features;

[0020] Step 22: Cross-scale entropy coding compression;

[0021] The CSEC network is used to process the two single-scale features generated in step 21. and Compression is performed to eliminate cross-scale redundancy between two features;

[0022] The CSEC network includes and Two modules, of which The module is used for Processing, including network units , , Specifically:

[0023] Network Unit The structure is: a quantizer Q + a super-prior network, wherein the super-prior network consists of a super-prior encoder. +Q+AE+AD+ ;in, For the hyper-prior encoder, AE and AD are the entropy encoder and entropy decoder, respectively. Advanced prior decoder, network unit Input for The output is ;

[0024] Network Unit The structure is: quantizer Q + context model + entropy parameter network The entropy parameter network The structure is Conv 1×1 + ReLU + Conv 1×1, entropy parameter network. Estimate Gaussian distribution parameters Used for Perform quantization and entropy coding; network units Input for Output for Network unit Input for and Output for and ;

[0025] Network Unit The structure is: quantizer Q + AE + AD; network unit Input for , and Output for ;

[0026] The module is used for Processing, including network units , and Specifically:

[0027] Network Unit The structure is: a quantizer Q + a super-prior network, wherein the super-prior network consists of a super-prior encoder. +Q+AE+AD+Concat+ ; where Concat is a concatenation function used to concatenate... The result after AD decoding in the super-prior network and The results after AD decoding are concatenated, and the network unit is... Input for The output is ;

[0028] Network Unit The structure is: quantizer Q + context model + entropy parameter network Network Unit Input for Output for Network unit Input for and Output for and ;

[0029] Network Unit The structure is: quantizer Q + AE + AD; network unit Input for , and Output for ;

[0030] Step 23: Hierarchical feature reconstruction;

[0031] The HFR network is used to process the two single-scale features after compression and decoding. and This restores the features to high quality, encompassing multiple scales. ;

[0032] The HFR network consists of network units. , , and The components are connected sequentially; specifically:

[0033] Network Unit The structure is: Attention + ResBlock + ResBlock↑ + ResBlock + ResBlock↑; the structure of ResBlock↑ is: the main path is Conv↑ + Leaky ReLU + Conv↑ + IGDN; the bypass is: Conv↑, and the final output is the sum of the main path and the bypass; the structure of Conv↑ is: Conv + PixelShuffle 2↑; network unit Input for The output is ;

[0034] Network Unit The structure is: Attention + ResBlock + ResBlock↑ + FeatureMixing; the input of FeatureMixing is... and The network structure is: Conv 2↓ + Leaky ReLU + Concat + Conv + Leaky ReLU, and the final output is the sum of the network output and the... The sum; network unit Input for and The output is ;

[0035] remember The output after Attention+Conv is: ,remember The output after Attention+Conv is: ,;

[0036] Network Unit The structure is: ResBlock + ResBlock↑ + Attention + ResBlock + ResBlock↑ + ResBlock↑ + Conv; Network unit Input for + The output is ;

[0037] Network Unit The structure is: ResBlock + ResBlock↑ + Attention + ResBlock + ResBlock↑ + FeatureMixing; network unit Input for + and The output is ;

[0038] The final output of the cross-scale feature compression network is , , , ;

[0039] Step 3: Network optimization and training;

[0040] The hierarchical feature fusion network, the cross-scale entropy coding network, and the hierarchical feature reconstruction network are jointly trained end-to-end. The training process is performed by optimizing a rate-distortion loss function, defined as follows:

[0041] ,

[0042] in, This represents the bitrate generated during encoding in step S2. It is a constant used to measure the degree of compression. The distortion term is a weighted sum of the task loss and the feature reconstruction loss, specifically defined as:

[0043] ,

[0044] in, Represents the use of reconstruction features Task loss resulting from performing downstream machine vision tasks. Original features With reconstruction features The mean square error between them, and It is a weighting factor that balances the two parts of the loss; the task loss expression is:

[0045] ,

[0046] The classification loss is used to evaluate the model's performance on classification tasks. It is obtained by calculating the difference between the model's predicted class probabilities and the true labels, using the cross-entropy loss function:

[0047] ,

[0048] in, For the real category, It is the predicted class probability;

[0049] The regression loss is used to evaluate the difference between the model's predicted continuous values ​​(such as the coordinates of the bounding box) and the true values. The regression loss function is the mean squared error, and its expression is:

[0050] ,

[0051] in, The total number of samples, For the first The true category of each sample For the model to the first Predicted values ​​for each sample;

[0052] The pre-regression loss is used to adjust intermediate features and prevent overfitting. The L2 norm is used to regularize the features, and its expression is:

[0053] ,

[0054] in, It is the L2 norm. f is the regularization coefficient. Features extracted by the network;

[0055] The object detection loss is used in object detection tasks to evaluate whether the model's predicted region contains an object. It calculates the probability that the model predicts an object in each candidate region, using the binary cross-entropy loss, and its expression is:

[0056] ,

[0057] in, Indicates whether the area contains an object. The probability of the object's existence as predicted by the model;

[0058] A well-trained cross-scale feature compression network is obtained through training.

[0059] Step 4: Network Reasoning and Goal Achievement:

[0060] A pre-trained cross-scale feature compression network is used to apply its output features to downstream machine vision tasks, such as object detection and segmentation.

[0061] The beneficial effects of this invention are:

[0062] This invention constructs an end-to-end cross-scale feature compression framework consisting of a hierarchical feature fusion network, a cross-scale entropy coding network, and a hierarchical feature reconstruction network. This framework achieves efficient compression and reconstruction of multi-scale features while ensuring performance in downstream machine vision tasks such as object detection and instance segmentation. In the feature pyramid output stage, based on the importance of each scale to the downstream task, multi-scale features are grouped and fused into two single-scale features. This fully exploits and eliminates redundancy between different scales while effectively reducing the dimensionality of feature data without significantly sacrificing key task information, thus creating conditions for subsequent compression. In the entropy coding stage, the cross-scale entropy coding network introduces latent variables and repetitions from low-level features. The reconstruction features, as prior information for high-level features, are used to jointly model and adaptively estimate the Gaussian parameters of the two feature paths, effectively improving the accuracy of the probabilistic model and thus significantly reducing the feature bit rate while ensuring task performance. At the decoding end, the hierarchical feature reconstruction network recovers the structurally complete multi-scale feature pyramid step by step through upsampling, residual and attention modules. Downstream tasks can directly call the reconstructed features without reconstructing the image and repeating feature extraction, which significantly reduces the computational overhead on the decoding side. In summary, this invention, by making full use of the cross-scale correlation of multi-scale features, achieves the goal of maintaining higher accuracy of machine vision tasks at a lower bit rate, thereby effectively improving the overall feature compression efficiency and inference performance. Attached Figure Description

[0063] Figure 1 This is the cross-scale entropy coding network proposed in this embodiment of the invention.

[0064] Figure 2 This refers to the hierarchical feature fusion network proposed in this embodiment of the invention.

[0065] Figure 3 This refers to the hierarchical feature reconstruction network proposed in this embodiment of the invention.

[0066] Figure 4 The embodiments and comparative examples of this invention demonstrate the performance of the target detection task on the same dataset.

[0067] Figure 5 The embodiments and comparative examples of this invention demonstrate the performance of target segmentation tasks on the same dataset. Detailed Implementation

[0068] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings and examples.

[0069] This example includes the following steps:

[0070] Step 1. Multi-scale feature extraction;

[0071] Input image The input is fed into a pre-trained feature pyramid network to extract feature maps containing multiple scales. The multi-scale features Feature information contained at different scales Each feature map at each scale has the same number of channels, but different spatial resolutions;

[0072] Step 2. Network setup;

[0073] The construction of a cross-scale feature compression network consists of three parts: the first part is a hierarchical feature fusion (HFF) network, the second part is a cross-scale entropy coding (CSEC) network, and the third part is a hierarchical feature reconstruction (HFR) network. The process of constructing the cross-scale feature compression network for processing multi-scale features is as follows:

[0074] Hierarchical feature fusion;

[0075] Construct and apply a hierarchical feature fusion network to integrate the multi-scale features extracted in step 1. Based on their importance to machine vision tasks, features are grouped and fused into two scales. and The specific implementation is as follows:

[0076] The hierarchical feature fusion network consists of network units. , , and Composed of sequential connections, such as Figure 2 As shown, specifically:

[0077] Network Unit The residual downsampling block has the following structure: ResDown + ResBlock. ResDown consists of a backbone and a bypass structure. The backbone is Conv 3×3 + Norm + ReLU + Conv 3×3, and the bypass is Conv 1×1. The input of Conv 3×3 in the backbone passes through the bypass Conv 1×1 and is then fused with the output of the backbone to obtain the output of ResDown. The main structure of ResBlock is Conv 3×3 + ReLU + Conv 3×3 + ReLU. The output of the main structure is added to the input to obtain the ResBlock output. (Network unit) Input Features of pyramids The output is ;

[0078] Network Unit The structure is: Concat( The network unit consists of ResDown, ResBlock, and Attention modules, where the Attention module is the attention module itself. Input for The output is , Indicates by , Fusion of low-level single-scale features;

[0079] Network Unit The structure is: ResDown + ResBlock; network unit Input for The output is ;

[0080] Network Unit The structure is: Concat( ) + Conv 3×3 + Attention; Network Unit Input for The output is , Indicates by , The fusion of high-level single-scale features.

[0081] Features with minimal impact on task performance and high spatial resolution and The first single-scale feature is generated through HFF network fusion. Features that significantly impact task performance and have low spatial resolution and A second single-scale feature is generated through HFF network fusion. The final output of this network is , , as input to the cross-scale entropy coding network.

[0082] Cross-scale entropy coding compression;

[0083] Construct and apply a cross-scale entropy coding network, such as Figure 1 As shown, the two single-scale features generated in step 2... and Compression is performed to eliminate cross-scale redundancy between two features. The network contains... and Two modules. The specific implementation of the cross-scale entropy coding network is as follows:

[0084] The CSEC network includes and Two modules, of which The module is used for Processing, including network units , , Specifically:

[0085] Network Unit The structure is: a quantizer Q + a super-prior network, wherein the super-prior network consists of a super-prior encoder. +Q+AE+AD+ ;in, For the hyper-prior encoder, AE and AD are the entropy encoder and entropy decoder, respectively. Advanced prior decoder, network unit Input for The output is ;

[0086] Network Unit The structure is: quantizer Q + context model + entropy parameter network The entropy parameter network The structure is Conv 1×1 + ReLU + Conv 1×1, entropy parameter network. Estimate Gaussian distribution parameters Used for Perform quantization and entropy coding; network units Input for Output for Network unit Input for and Output for and ;

[0087] Network Unit The structure is: quantizer Q + AE + AD; network unit Input for , and Output for ;

[0088] The module is used for Processing, including network units , and Specifically:

[0089] Network Unit The structure is: a quantizer Q + a super-prior network, wherein the super-prior network consists of a super-prior encoder. +Q+AE+AD+Concat+ ; where Concat is a concatenation function used to concatenate... The result after AD decoding in the super-prior network and The results after AD decoding are concatenated, and the network unit is... Input for The output is ;

[0090] Network Unit The structure is: quantizer Q + context model + entropy parameter network Network Unit Input for Output for Network unit Input for and Output for and ;

[0091] Network Unit The structure is: quantizer Q + AE + AD; network unit Input for , and Output for .

[0092] use Module for features Encoding is performed. Gaussian distribution parameters are estimated using its built-in super-prior network and context model. And guided by this parameter, Quantization and entropy coding are performed to obtain the compressed bitstream and the decoded features. This process generates intermediate latent variables. ;use Module for features Encoding is performed. During the encoding process, [the following is utilized]: Generated latent variables and decoding features As guiding information, to more accurately estimate the encoding Gaussian distribution parameters Finally, the decoded features are obtained.

[0093] The final output of this network is , , which serves as the input to the hierarchical feature reconstruction network.

[0094] Hierarchical feature reconstruction;

[0095] Construct a hierarchical feature reconstruction network to combine the two single-scale features after compression and decoding. and This restores the features to high quality, encompassing multiple scales. The specific implementation is as follows:

[0096] The hierarchical feature reconstruction network consists of network units. , , and Composed of sequential connections, such as Figure 3 As shown; specifically:

[0097] Network Unit The structure is: Attention + ResBlock + ResBlock↑ + ResBlock + ResBlock↑; the structure of ResBlock↑ is: the main path is Conv↑ + Leaky ReLU + Conv↑ + IGDN; the bypass is: Conv↑, and the final output is the sum of the main path and the bypass; the structure of Conv↑ is: Conv + PixelShuffle 2↑; network unit Input for The output is ;

[0098] Network Unit The structure is: Attention + ResBlock + ResBlock↑ + FeatureMixing; the input of FeatureMixing is... and The network structure is: Conv 2↓ + Leaky ReLU + Concat + Conv + Leaky ReLU, and the final output is the sum of the network output and the... The sum; network unit Input for and The output is ;

[0099] remember The output after Attention+Conv is: ,remember The output after Attention+Conv is: ,;

[0100] Network Unit The structure is: ResBlock + ResBlock↑ + Attention + ResBlock + ResBlock↑ + ResBlock↑ + Conv; Network unit Input for + The output is ;

[0101] Network Unit The structure is: ResBlock + ResBlock↑ + Attention + ResBlock + ResBlock↑ + FeatureMixing; network unit Input for + and The output is .

[0102] feature After a series of upsampling, residual, and attention modules, high-level multi-scale features (with lower spatial resolution) are reconstructed. and Combined features and The information is processed through a series of similar upsampling, residual, and attention modules to reconstruct high-level multi-scale features (with high spatial resolution). and The reconstructed multi-scale features are obtained. .

[0103] The final output of this network is , , , It is used for direct calls by downstream machine vision tasks.

[0104] Step 3. Network optimization and training;

[0105] The hierarchical feature fusion network, cross-scale entropy coding network, and hierarchical feature reconstruction network are jointly trained end-to-end. The training process is performed by optimizing a rate-distortion loss function, which is defined as follows:

[0106]

[0107] in, This indicates the bitrate generated during encoding in step 2. It is a constant used to measure the degree of compression. The distortion term is a weighted sum of the task loss and the feature reconstruction loss, and is specifically defined as follows:

[0108]

[0109] in, Represents the use of reconstruction features Losses incurred from performing downstream machine vision tasks Original features With reconstruction features The mean square error between them, and It is a weighting factor that balances the two parts of the loss. The task loss expression is:

[0110] ,

[0111] The classification loss is used to evaluate the model's performance on classification tasks. It is obtained by calculating the difference between the model's predicted class probabilities and the true labels, using the cross-entropy loss function:

[0112] ,

[0113] in, For the real category, It is the predicted class probability;

[0114] The regression loss is used to evaluate the difference between the continuous values ​​predicted by the model and the true values. The regression loss function is the mean squared error, and its expression is:

[0115] ,

[0116] in, The total number of samples, For the first The true category of each sample For the model to the first Predicted values ​​for each sample;

[0117] The pre-regression loss is used to adjust intermediate features and prevent overfitting. The L2 norm is used to regularize the features, and its expression is:

[0118] ,

[0119] in, It is the L2 norm. f is the regularization coefficient. Features extracted by the network;

[0120] The object detection loss is used in object detection tasks to evaluate whether the model's predicted region contains an object. It calculates the probability that the model predicts an object in each candidate region, using the binary cross-entropy loss, and its expression is:

[0121] ,

[0122] in, Indicates whether the area contains an object. The probability of the object's existence as predicted by the model;

[0123] A well-trained cross-scale feature compression network is obtained through training.

[0124] Step 4: Network Reasoning and Goal Achievement:

[0125] A pre-trained cross-scale feature compression network is used to apply its output features to downstream machine vision tasks, such as object detection and segmentation.

[0126] Based on the above technical solution, the training set and test set used in this embodiment are both COCO 2017, and six points with different compression qualities are selected for comparison; four methods are used to compare with the method in the example of this invention: 1) Comparative example 1 uses SPL-2024[1] for feature compression, 2) Comparative example 2 uses TCSVT-2024[2] for feature compression, 3) Comparative example 3 uses VVC[3] for image compression, 4) Comparative example 4 uses MPEG-VCM for feature compression. The test results of the target detection and segmentation tasks corresponding to the four comparison methods and the method proposed in this invention are as follows: Figure 4 , Figure 5 As shown, the horizontal axis represents bpp (bits per pixel), which indicates the number of bits required to transmit each pixel in the image, and the vertical axis represents mAP (mean average precision), which is used to measure the overall accuracy of object detection and segmentation tasks. The six points corresponding to each method represent images of different qualities, with image quality increasing sequentially from left to right.

[0127] Depend on Figure 4and Figure 5 As can be seen, across the entire test bitrate range, the mAP–bpp curve of this invention consistently outperforms that of Comparative Examples 1–4: higher mAP is achieved at the same bpp; a lower bpp is required for the same mAP target, with the advantage being particularly significant in the low bitrate range. This demonstrates that this invention can maintain higher detection and segmentation performance at a lower bitrate, and the experimental results verify the effectiveness and superiority of the method of this invention.

[0128] In summary, the cross-scale feature compression method for machine vision tasks proposed in this invention exhibits excellent performance. In the COCO 2017 image set, compared with other methods, it can maintain better machine vision task performance at a lower bpp.

[0129] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A cross-scale feature compression method for machine vision tasks, characterized in that, Includes the following steps: Step 1: Multi-scale feature extraction; The input image is input into a pre-trained feature pyramid network (FPN) to extract multi-scale features containing multiple scales , , , Pyramid features represent feature map information at different scales. Feature maps at each scale have the same number of channels but different spatial resolutions. Step 2: Network setup; The construction of a cross-scale feature compression network consists of three parts: the first part is a hierarchical feature fusion (HFF) network, the second part is a cross-scale entropy coding (CSEC) network, and the third part is a hierarchical feature reconstruction (HFR) network. The process of constructing the cross-scale feature compression network for processing multi-scale features is as follows: Step 21: Layered feature fusion; The HFF network is used to fuse the multi-scale features extracted in step 1 According to the importance of the machine vision task, the features are grouped and fused into two scales respectively and The specific implementation is as follows: The HFF network is composed of network units , , and are sequentially connected, specifically: Network Unit The residual downsampling block has the following structure: ResDown + ResBlock. ResDown consists of a backbone and a bypass structure. The backbone is Conv 3×3 + Norm + ReLU + Conv 3×3, and the bypass is Conv 1×1. The input of Conv 3×3 in the backbone passes through the bypass Conv 1×1 and is then fused with the output of the backbone to obtain the output of ResDown. The main structure of ResBlock is Conv 3×3 + ReLU + Conv 3×3 + ReLU. The output of the main structure is added to the input to obtain the ResBlock output. (Network unit) Input Features of pyramids The output is ; Network Unit The structure is: Concat( The network unit consists of ResDown, ResBlock, and Attention modules, where the Attention module is the attention module itself. Input for The output is , Indicates by , Fusion of low-level single-scale features; Network Unit The structure is: ResDown + ResBlock; network unit Input for The output is ; Network Unit The structure is: Concat( ) + Conv 3×3 + Attention; Network Unit Input for The output is , Indicates by , The fusion of high-level single-scale features; Step 22: Cross-scale entropy coding compression; The CSEC network is used to process the two single-scale features generated in step 21. and Compression is performed to eliminate cross-scale redundancy between two features; The CSEC network includes and Two modules, of which The module is used for Processing, including network units , , Specifically: Network Unit The structure is: a quantizer Q + a super-prior network, wherein the super-prior network consists of a super-prior encoder. +Q+AE+AD+ ;in, For the hyper-prior encoder, AE and AD are the entropy encoder and entropy decoder, respectively. Advanced prior decoder, network unit Input for The output is ; Network Unit The structure is: quantizer Q + context model + entropy parameter network The entropy parameter network The structure is Conv 1×1 + ReLU + Conv 1×1, entropy parameter network. Estimate Gaussian distribution parameters Used for Perform quantization and entropy coding; network units Input for Output for Network unit Input for and Output for and ; Network Unit The structure is: quantizer Q + AE + AD; network unit Input for , and Output for ; The module is used for Processing, including network units , and Specifically: Network Unit The structure is: a quantizer Q + a super-prior network, wherein the super-prior network consists of a super-prior encoder. +Q+AE+AD+Concat+ ; where Concat is a concatenation function used to concatenate... The result after AD decoding in the super-prior network and The results after AD decoding are concatenated, and the network unit is... Input for The output is ; Network Unit The structure is: quantizer Q + context model + entropy parameter network Network Unit Input for Output for Network unit Input for and Output for and ; Network Unit The structure is: quantizer Q + AE + AD; network unit Input for , and Output for ; Step 23: Hierarchical feature reconstruction; The HFR network is used to process the two single-scale features after compression and decoding. and This restores the features to high quality, encompassing multiple scales. ; The HFR network consists of network units. , , and The components are connected sequentially; specifically: Network Unit The structure is: Attention + ResBlock + ResBlock↑ + ResBlock + ResBlock↑; the structure of ResBlock↑ is: the main path is Conv↑ + Leaky ReLU + Conv↑ + IGDN; the bypass path is: Conv↑, and the final output is the sum of the main path and the bypass path; the structure of Conv↑ is: Conv + PixelShuffle 2↑; network unit Input for The output is ; Network Unit The structure is: Attention + ResBlock + ResBlock↑ + FeatureMixing; the input of FeatureMixing is... and The network structure is: Conv 2↓ + Leaky ReLU + Concat + Conv + Leaky ReLU, and the final output is the sum of the network output and the... The sum; network unit Input for and The output is ; remember The output after Attention+Conv is: ,remember The output after Attention+Conv is: ,; Network Unit The structure is: ResBlock + ResBlock↑ + Attention + ResBlock + ResBlock↑ + ResBlock↑ + Conv; Network unit Input for + The output is ; Network Unit The structure is: ResBlock + ResBlock↑ + Attention + ResBlock + ResBlock↑ + FeatureMixing; network unit Input for + and The output is ; The final output of the cross-scale feature compression network is , , , ; Step 3: Network optimization and training; The hierarchical feature fusion network, the cross-scale entropy coding network, and the hierarchical feature reconstruction network are jointly trained end-to-end. The training process is performed by optimizing a rate-distortion loss function, defined as follows: ， in, This represents the bitrate generated during encoding in step S2. It is a constant used to measure the degree of compression. The distortion term is a weighted sum of the task loss and the feature reconstruction loss, specifically defined as: ， in, Represents the use of reconstruction features Task loss resulting from performing downstream machine vision tasks. Original features With reconstruction features The mean square error between them, and It is a weighting factor that balances the two parts of the loss; the task loss expression is: ， The classification loss is used to evaluate the model's performance on classification tasks. It is obtained by calculating the difference between the model's predicted class probabilities and the true labels, using the cross-entropy loss function: ， in, For the real category, It is the predicted class probability; The regression loss is used to evaluate the difference between the model's predicted continuous values ​​and the true values. The regression loss function is the mean squared error, and its expression is: ， in, The total number of samples, For the first The true category of each sample For the model to the first Predicted values ​​for each sample; The pre-regression loss is used to adjust intermediate features and prevent overfitting. The L2 norm is used to regularize the features, and its expression is: ， in, It is the L2 norm. f is the regularization coefficient. Features extracted by the network; The object detection loss is used in object detection tasks to evaluate whether the model's predicted region contains an object. It calculates the probability that the model predicts an object in each candidate region, using the binary cross-entropy loss, and its expression is: ， in, Indicates whether the area contains an object. The probability of the object's existence as predicted by the model; A well-trained cross-scale feature compression network is obtained through training. Step 4: Network Reasoning and Goal Achievement A pre-trained cross-scale feature compression network is used, and its output features are applied to downstream machine vision tasks.

Images