Normalization flow based compression image enhancement method for machine vision tasks

Abstract

The application discloses a normalized flow-based compressed image enhancement method for machine vision tasks, and is specifically implemented according to the following steps: step 1, obtaining a decoding side latent feature and a decoding image; step 2, obtaining a task prior feature by taking the decoding image as input, performing feature fusion and extraction on the task prior feature, the decoding side latent feature and the decoding image, and obtaining a conditional feature; step 3, constructing a conditional normalization flow model based on the conditional feature; step 4, designing a loss function; step 5, training the conditional normalization flow model constructed in step 3 by using the loss function, and updating model parameters; and step 6, performing inference testing by using the conditional normalization flow model obtained in step 5. The application solves the problems of the prior art, such as mispositioning of an optimization target, limited distortion modeling capability, unstable generation quality and poor universality.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a normalized flow-based compressed image enhancement method for machine vision tasks. Background Technology

[0002] Currently, methods to address the degradation of decoded image quality mainly include traditional post-processing, deep learning-based restoration, end-to-end learning-based compression, machine vision-oriented compression, and normalized streaming image restoration. Traditional post-processing includes denoising filtering, sharpening, and super-resolution reconstruction, which restore image quality through signal processing. However, it cannot model complex distortion distributions, and the optimization goals are disconnected from actual task requirements. Deep learning-based restoration includes image restoration based on Generative Adversarial Networks (GANs) and reconstruction based on diffusion models. It utilizes generative models to learn degradation-sharpening mappings, but GAN training is unstable and prone to artifacts; diffusion models have slow inference speeds. End-to-end learning-based compression includes ELIC, TCM, and DCAE, which jointly optimize compression coding and image reconstruction, but require retraining the entire encoder and cannot reuse existing compression models. Machine vision-oriented compression involves joint optimization of task loss and rate loss, introducing downstream task loss into rate-distortion optimization, but requires access to both the encoder and the task network; changing tasks requires retraining the entire network model. Normalized flow image restoration includes flow-based super-resolution and image enhancement. Its principle is to model the image distribution using reversible transformations, but it is only geared towards human visual quality (PSNR / SSIM) and is not optimized for machine vision tasks.

[0003] In summary, the existing technical solutions have the following common defects: (1) Optimization target misalignment: Traditional post-processing and deep learning recovery methods use fidelity indicators such as PSNR / SSIM as optimization targets, which are not aligned with the actual needs of downstream machine vision tasks (such as salient object detection, semantic segmentation, etc.). High PSNR / SSIM does not necessarily lead to high detection accuracy, and may even impair task performance due to excessive edge smoothing.

[0004] (2) Limited distortion modeling capability: Compression distortion has structural characteristics (such as block effect, ringing effect), spatial non-uniformity, and complex distribution. Methods based on deterministic mapping (such as CNN direct regression) learn a one-to-one mapping from the decoded image to the original image, which makes it difficult to capture the multimodal conditional distribution characteristics of compression distortion, resulting in poor recovery performance under low bit rate conditions.

[0005] (3) Unstable generation quality: Although GAN-based methods can improve visual quality, they have problems such as unstable training and pattern collapse, which can easily produce artifacts and false textures, and may introduce misleading information and interfere with the judgment of downstream machine vision tasks.

[0006] (4) High universality and high deployment cost: End-to-end learning compression methods need to be embedded in the compression model for retraining and cannot be deployed as an independent module on the existing compression system; compression methods for machine vision need to access both the encoder and the task network at the same time, and the entire system needs to be retrained when the downstream task is changed, resulting in high deployment and maintenance costs.

[0007] (5) Technological gap: There is currently no compression distortion compensation scheme that can jointly optimize the normalizing flow with the loss of downstream machine vision tasks and deploy it as an independent plug-and-play post-processing module.

[0008] Therefore, there is an urgent need for a plug-and-play, task-aware compression distortion compensation method with the ability to model complex distortion distributions, in order to fill the gaps in the existing technologies mentioned above. Summary of the Invention

[0009] The purpose of this invention is to provide a normalized flow-based compressed image enhancement method for machine vision tasks, which solves the problems of existing technologies such as target misalignment optimization, limited distortion modeling capabilities, unstable generation quality, and poor versatility.

[0010] To achieve the above objectives, the technical solution adopted by this invention is: a normalized flow-based compressed image enhancement method for machine vision tasks, specifically implemented according to the following steps: Step 1: Obtain latent features and decoded image on the decoding side; Step 2: Obtain task prior features by taking the decoded image as input, and perform feature fusion and extraction on the task prior features, decoding-side latent features and decoded image to obtain conditional features; Step 3: Based on the conditional features, construct a conditional normalized flow model; Step 4: Design the loss function; Step 5: Use the loss function to train the conditionally normalized flow model constructed in Step 3 and update the model parameters. Step 6: Use the conditional normalized flow model obtained in Step 5 to perform inference testing.

[0011] The technical solution of the present invention also has the following characteristics: Step 1 specifically includes: Step 1.1, extract the original image Input compression encoder Ecomp( The bitstream is generated after quantization and arithmetic encoder. Step 1.2: Process the bitstream obtained in Step 1.1 using an arithmetic decoder to obtain the latent features on the decoding side. , dimension M × ( H / 16) × ( W / 16), of which M The number of latent feature channels in the compressed model; Step 1.3, at the decoding end, latent features on the decoding side. via decoder Dcomp( Reconstruct to obtain the decoded image. =Dcomp( ).

[0012] Step 2 specifically involves: decoding the 3-channel image. Decoding-side latent features of channel 1 upsampling features With 1-channel task prior feature SOD ( By concatenating the features along the channel dimension, a 5-channel combined feature is obtained. , SOD ( Afterwards, the conditional features are encoded by a convolutional network in the conditional feature encoding module to output the conditional features. ,in, This is the output feature dimension.

[0013] The specific steps for constructing the conditionally normalized flow model in step 3 are as follows: Step 3.1, the normalized flow model Φ is derived from... L It is formed by layer transformation and stacking. L For each layer, the integer is a positive integer greater than or equal to 2. Each layer consists of two sub-layers connected in series: an "Affine Coupling Layer (ACL)" and a "Reversible Linear Transformation". The overall forward transformation is represented as:

[0014] in r Input features (distorted residuals or enhanced target features). F The conditional features output by the conditional feature encoding module. z For latent variables, the inverse transformation is:

[0015] in The generated compensation residual is used to subsequently overlay the image with the decoded image to obtain the compensated image; During the training phase, a forward transformation is used to transform the input target residual. r Mapping to latent variablesz It approximates the standard Gaussian distribution by maximizing the likelihood constraint; in the inference stage, it employs an inverse transformation to sample latent variables from the standard Gaussian distribution. z sample The inverse transformation generates the compensation residual. To compensate for the impact of compression distortion on the performance of downstream tasks; Step 3.2, each ACL will share the same conditional features. F Mapped to a scaling factor specific to this layer. c and offset factor β It consists of the following components: Multilayer perceptron: Consists of two fully connected layers and a GELU activation function, which integrates conditional features. F Mapped to intermediate representation: ,in, W 1, W 2 represents the learnable weight matrix. b 1, b 2 represents the bias parameter; Spatial attention mechanism: A convolution-based spatial attention module is used to apply spatial dimension attention modulation to the intermediate representation, enabling the model to focus on spatially significant locations with distortion. ,in This is a temperature coefficient used to control the dynamic range of scale parameters; Zero-initialization strategy: The weights of the last layer of the Multilayer Perceptron (MLP) are initialized to zero, so that in the early stages of training... c ≈ 1, β ≈ 0, at which point ACL is approximately an identity transformation, which is beneficial for stable training; In ACL, given input features scaling factor and offset factor The forward affine transformation is defined as:

[0016] in To represent element-wise multiplication, the corresponding inverse transformation is:

[0017] in This represents element-wise division; this design guarantees the reversibility of the transformation—given a reverse input. It can accurately restore Meanwhile, it allows the conditional features to adaptively modulate the residual transformation process in each flow layer, thereby achieving fine-grained spatial adaptive compensation. Step 3.3: For reversible linear transformation layers, each transformation first applies a reversible linear transformation to the input features. h Perform linear mixing along the channel dimension: h' = · h + b in It is an invertible weight matrix. b For bias vectors, D For the input feature dimension, D = × P × P ,in Input the number of channels. P Let the block size be denoted by ; the logarithmic determinant of this transformation is:

[0018] The role of reversible linear transformation is to facilitate information exchange between feature channels and enhance the expressive power of normalized flow.

[0019] The loss function designed in step 4 is as follows: Step 4.1, design the negative log-likelihood loss; During the training phase, the compression distortion residual is first calculated: the original image... x With decoded images Perform interpolation to obtain the distortion difference. r = x - This is referred to as "target compensation residual"; the target residual... r according to P × P The block size is expanded into feature vectors, fed into a normalized stream to perform a forward transformation, and the negative log-likelihood loss is calculated:

[0020] The role of negative log-likelihood loss is to train the normalized flow model to learn the conditional probability distribution of the compressed distortion residual under given conditional information, and to normalize the complex distortion distribution to a standard Gaussian distribution. Step 4.2, design pixel-level detection loss; During the inference phase, latent variables are sampled from a standard Gaussian distribution. Compensation residuals are generated through inverse reversible transformation. It is then superimposed pixel by pixel onto the decoded image. To obtain an enhanced image = + The enhanced image is fed into the SOD network to obtain the enhanced saliency probability map. = Sigmoid(SOD( )); Calculate the augmented probability map and the true label. mask L1 loss between:

[0021] This loss is optional and is used to provide pixel-level fidelity supervision; Step 4.3, design the consistency loss for detection; The logarithmic graph of the output of the enhanced SOD network With real labels mask Calculate the combined SOD loss between:

[0022] The definitions of each item are as follows: Weighted binary cross-entropy loss An edge-aware weighted strategy is employed, assigning higher weights to salient target edge regions, making the model more focused on compensation accuracy at boundaries; the weight map is calculated from the local average pooling difference of the true labels. The weighted BCE loss is:

[0023] Weighted intersection and combination loss :

[0024] Loss (L) F-measure ):

[0025] in TP = Σ( prediction ⊙ mask ), FN = Σ( mask ) - TP , FP = Σ( prediction ) - TP , ; Step 4.4, Design the total loss function; The above losses are summed according to preset weights to obtain the total training loss:

[0026] in , , The preset weighting coefficients are used; the preferred configuration is as follows: = 5×10^ -4, = 50, = 0.4; The total loss is dominated by the task loss, while the NLL loss plays an auxiliary regularization role, guiding the latent space to follow a Gaussian distribution.

[0027] Step 5 specifically involves: Step 5.1, Data Preparation: Sample raw images from the training set. x and corresponding pixel-level saliency annotations mask ;Will x The latent features on the decoding side are obtained by compressing the data using a compression model and then decoding it. and decoding images ; Step 5.2, Conditional Feature Extraction: [The text abruptly ends here, likely due to an incomplete sentence or Input the frozen SOD network to obtain the SOD logarithmic graph (SOD). SOD ( ), decoding-side latent features Upsampling features and Channels are concatenated and encoded by a conditional feature encoding module to obtain conditional features. F ; Step 5.3, Target Residual Calculation: Calculate the original image... x With decoded images Perform the difference calculation; calculate the target residual. r = x - ;Will r according to P × P The block size is expanded to D 3D feature vector; Step 5.4, Forward Transformation and NLL Calculation: Calculate the target residual. r Input the normalized flow model Φ, with conditional features F To guide execution L Layer forward reversible transformation yields latent variables z = Φ( r | F ); Calculate the negative log-likelihood loss using the formula. L NLL ; Step 5.5, Inverse Transformation and Task Loss Calculation: From Standard Gaussian Distribution Randomly sampled latent variables in (0, I) z sample Compensation residuals are generated through inverse reversible transformation. The compensated residual is superimposed pixel by pixel onto the decoded image. The enhanced image is then input into the SOD network to obtain: SOD en = SOD( + ),calculate L SOD loss; Step 5.6, Parameter Update: Based on total loss To optimize the objective, update the normalized flow model parameters θ. flow and conditional feature encoding module parameter θ encoder The parameters of the compressed model and the SOD model remain frozen and do not participate in gradient updates. Step 5.7, Iterative training: Repeat steps 5.1 to 5.6 until the preset number of training rounds is reached or the early stopping condition is triggered.

[0028] Step 6 specifically includes: Step 6.1, Input the compressed / decoded image to be processed. Extracting conditional features F ; Step 6.2, from the standard Gaussian distribution Randomly sampled latent variables in (0, I) ; Step 6.3: Generate the compensation residual through the inverse reversible transformation of the normalized flow: ; Step 6.4, compensate for the residual The compensated features are obtained by superimposing pixel-by-pixel onto the decoded image; in the SOD task scenario, the compensated SOD response is: SOD en = SOD( + ) ; Step 6.5: Feed the compensated features into the downstream task model to obtain the final prediction result.

[0029] In step 6: multiple random samplings can be performed to generate... K Different compensation results are obtained, and performance and robustness are further improved by integrating and averaging.

[0030] The beneficial effects of this invention are: (1) As an independent post-processing module, this invention can be deployed on the backend of any existing deep learning-based image compression algorithm without modifying the compressed bitstream or retraining the compression model. The experimentally verified compatible compression algorithms include DCAE and TCM. On these two compression models with significant structural differences, this invention exhibits stable and consistent SOD performance improvement, indicating that it has good cross-compression architecture generalization ability.

[0031] (2) The present invention achieves consistent improvement across all core evaluation metrics without introducing performance degradation in any metric. Comprehensive experiments on different bitrates (0.048~0.218 bpp) and different datasets (ORSSD, EORSSD, ORSI-4199) have verified the effectiveness and stability of the present invention.

[0032] (3) The generalization ability of this invention has been verified in the following dimensions: a) Generalization across compression algorithms: The normalized flow compensation model trained on DCAE can be directly deployed to the TCM compression backend without any retraining or parameter adjustment, and still achieves significant and consistent performance improvement on the downstream SOD task. This proves that the compensation ability of this invention is based on general distortion modeling, rather than overfitting to the artifact mode of a specific compression algorithm. b) Generalization across datasets: On three independent SOD test sets (ORSSD-200 images, EORSSD-600 images, ORSI-4199-2199 images), this invention shows stable performance improvement and is effective under different target sizes and scene complexities. c) Technical significance of generalization verification: The above results show that this invention extracts and compensates for the "loss of task-related information" that is common in the compression process, rather than the product of a specific compression architecture or a specific data distribution, thus verifying the feasibility of normalized flow as a general compensation framework for compression distortion. Attached Figure Description

[0033] Figure 1 This is an overall framework diagram of the present invention; Figure 2 This is the ACL block diagram in the normalized conditional flow model; Figure 3 This is a qualitative comparison result diagram of square saliency targets under the DCAE compressed benchmark; Figure 4 This is a qualitative comparison result diagram on a circular salient target under the DCAE compressed benchmark; Figure 5 This is a qualitative comparison result diagram under the TCM compression benchmark. Detailed Implementation

[0034] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] Example 1 like Figure 1 As shown, the present invention provides a normalized flow-based compressed image enhancement method for machine vision tasks, which is implemented according to the following steps: Step 1: Use any existing image compression algorithm to encode, compress, decode, and reconstruct the original image to obtain the decoded image and the latent features on the decoding side; Step 2, decode the image Input the pre-trained downstream task network to extract task prior features, and input the task prior features, decoding-side latent features, and decoded image into the conditional feature encoding network for feature fusion and extraction to obtain conditional features; Step 3: Based on the conditional features, construct a conditional normalized flow model; Step 4: Design the loss function; Step 5: Use the loss function to train the conditionally normalized flow model constructed in Step 3 and update the model parameters. Step 6: Use the conditional normalized flow model obtained in Step 5 to perform inference testing. Sample latent variables from the standard Gaussian distribution and input the latent variables into the trained conditional normalized flow model. Generate compensation residuals through its inverse transformation. Superimpose the compensation residuals onto the decoded image to obtain the compensated image. Input the compensated image into the downstream task network to obtain the final prediction result.

[0036] Compared with the prior art, the present invention has the following advantages: (1) Plug and play capability.

[0037] This invention, as a standalone post-processing module, can be deployed on the backend of any existing deep learning-based image compression algorithm without modifying the compressed bitstream or retraining the compression model. For example... Figure 3 , Figure 4 and Figure 5 As shown, the experimentally validated compatible compression algorithms include DCAE and TCM. On both of these compression models with significantly different structures, this invention demonstrates a stable and consistent SOD performance improvement, indicating its good generalization ability across compression architectures.

[0038] Figure 3 This is a qualitative comparison result diagram on a square saliency target under the DCAE compressed benchmark. Figure 4 This is a qualitative comparison result diagram on a circular salient target under the DCAE compression benchmark.

[0039] Figure 5 Using TCM as the baseline compression algorithm, the test results on two images in the ORS-4199 test set are analyzed. From left to right, the columns are: the original image with true contour annotation (green), the TCM baseline prediction result, the prediction result enhanced by the Flow-RSIC method of this invention, and the pixel-by-pixel improvement image.

[0040] Experimental data: Using DCAE as the baseline compression algorithm, in lAt an extremely low bitrate of 0.0018 (approximately 0.048 bpp), this invention reduced MAE from 0.0108 to 0.0075 (a reduction of 30.6%), improved S-measure from 0.9015 to 0.9162 (+1.6%), and increased maxF from 0.8415 to 0.8592 (+2.1%) on the EORSSD test set. Using TCM as the baseline compression algorithm, l At a bitrate of 0.0025, MAE decreased from 0.0101 to 0.0080 (a decrease of 20.8%), maxF increased from 0.8474 to 0.8597 (+1.5%), and S-measure increased from 0.9082 to 0.9161 (+0.9%).

[0041] (2) Improved task performance.

[0042] The quantitative evaluation results of the DCAE compressed model (λ=0.0018, bpp≈0.048) on the EORSSD test dataset, with salient object detection (SOD) as the downstream task, are as follows:

[0043] The above results demonstrate that the present invention achieves consistent improvements across all core evaluation metrics without introducing performance degradation in any metric. Comprehensive experiments on different bitrates (0.048~0.218 bpp) and different datasets (ORSSD, EORSSD, ORSI-4199) have verified the effectiveness and stability of the present invention.

[0044] (3) Verification of cross-compression architecture generality.

[0045] The generalization ability of this invention has been verified in the following dimensions: (a) Generalization across compression algorithms: The normalized flow compensation model trained on DCAE is directly deployed to the TCM compression backend without any retraining or parameter tuning, and still achieves significant and consistent performance improvements on downstream SOD tasks. This demonstrates that the compensation capability of this invention is based on general distortion modeling, rather than overfitting to the artifact patterns of a specific compression algorithm.

[0046] (b) Cross-dataset generalization: On three independent SOD test sets (ORSSD-200 images, EORSSD-600 images, ORSI-4199-2199 images), the present invention showed stable performance improvement and was effective under different target sizes and scene complexities.

[0047] (c) Technical significance of generalization verification: The above results show that the present invention extracts and compensates for the "loss of task-related information" that is common in the compression process, rather than the product of a specific compression architecture or a specific data distribution, thus verifying the feasibility of normalized flow as a general compensation framework for compression distortion.

[0048] In summary, the present invention has technical advantages in the following aspects: (1) In terms of theoretical advantages, normalized flow can accurately calculate likelihood probability, with clear optimization objectives and stable training; reversible transformation theoretically guarantees lossless information transmission and avoids feature degradation.

[0049] (2) In terms of expressive power, the invertible transformation can model complex nonlinear and multimodal distributions, which is better than the deterministic single mapping method; the Gaussian latent space supports random sampling and can generate diverse compensation results.

[0050] (3) In terms of plug-and-play functionality: No need to modify the original compression model or bitstream format; no need to retrain the compression encoder / decoder; deployment only requires loading the stream model weights, and the additional computational overhead is controllable.

[0051] (4) In terms of task orientation. Directly optimize the loss of downstream machine vision tasks, and the performance improvement directly serves the actual application needs; do not pursue textures that are "good-looking" to the human eye but "useless" to the machine.

[0052] (5) In terms of architectural versatility. It has been proven to have broad compatibility with DCAE and TCM; the 5-channel conditional input design is naturally adapted to any 1-channel output SOD / segmentation network.

[0053] (6) In terms of computational efficiency, the inference stage only requires one inverse transformation forward propagation, without the need for iterative optimization; compared with generative compensation methods such as GAN / diffusion models, the inference speed is fast and the determinism is high.

[0054] Example 2 like Figure 1 As shown, unlike Example 1, step 1 of the normalized flow-based compressed image enhancement method for machine vision tasks in Example 2 of the present invention specifically includes: Step 1.1, extract the original image Input compression encoder Ecomp( The bitstream is generated after quantization and arithmetic encoder. Step 1.2: Process the bitstream obtained in Step 1.1 using an arithmetic decoder to obtain the latent features on the decoding side. , dimension M × ( H / 16) × ( W / 16), of which MThe number of latent feature channels in the compressed model; Step 1.3, at the decoding end, latent features on the decoding side. via decoder Dcomp( Reconstruct and obtain the decoded image. =Dcomp( ).

[0055] Example 3 like Figure 1 As shown, unlike Example 2, step 2 of the normalized flow-based compressed image enhancement method for machine vision tasks in Example 3 specifically involves: decoding the image... (3 channels), latent features on the decoding side upsampling features (1 channel) and task prior features SOD ( (1 channel) concatenated along the channel dimension to obtain a 5-channel combined feature. , SOD ( The conditional features are encoded by a convolutional network using the conditional feature encoding module (SG-CFE) and output as conditional features. ,in Output feature dimension (preferably 128). Decoding-side latent features. After upsampling, it is fused with the above features (optional).

[0056] Multi-source conditional information is extracted from the compression / decoding end to guide the normalized stream in learning the distortion distribution. This invention supports any combination of the following three types of features:

[0057] Analysis of the role of the three types of features: (a) Decoding image features It provides complete appearance information of the compressed and reconstructed image and serves as a fundamental reference for distortion compensation.

[0058] (b) Decoding-side latent features It carries structured prior information from the compression process, including quantization step size, bitrate allocation pattern, and other information that helps estimate the statistical characteristics of the distortion distribution. When this feature is unavailable (such as when using traditional coding standards), it can be filled with zero tensors, and the method can still work normally, with only a slight performance decrease.

[0059] (c) Task prior features SOD( ): Providing the initial discrimination response of the compressed image on the downstream task network, so that the compensation process focuses on the task-related distortion areas (such as the boundaries of salient targets, small target areas, etc.), is the key to realizing "task perception".

[0060] Example 4 Combination Figure 2 Unlike Example 3, in Example 4, step 3 of the compressed image enhancement method based on normalized flow for machine vision tasks of the present invention, the construction of the conditional normalized flow model specifically involves: Step 3.1 Overall structure of the model.

[0061] The normalized flow model Φ is derived from L Layer transformation stacking ( L It is a positive integer greater than or equal to 2, preferably L =12), each layer consists of two sublayers connected in series: an "affine coupling layer (ACL)" and a "reversible linear transformation". The overall forward transformation can be represented as:

[0062] in r Input features (distorted residuals or enhanced target features). F Let z be the conditional feature output by the conditional feature encoding module, and z be a latent variable. The inverse transformation is:

[0063] in: The generated compensation residual is used to subsequently overlay the decoded image to obtain the compensated image.

[0064] During the training phase, a forward transformation is used to transform the input target residual. r The latent variable z is mapped and its distribution is approximated to a standard Gaussian distribution by maximizing the likelihood constraint; in the inference phase, an inverse transformation is used to sample the latent variable from the standard Gaussian distribution. The inverse transformation generates the compensation residual. This is to compensate for the impact of compression distortion on the performance of downstream tasks.

[0065] Step 3.2 Affine coupling layer design.

[0066] Conditional characteristics shared by each ACL layer F Mapped to a scaling factor specific to this layer. c and offset factor β It consists of the following components: (a) MLP: Consists of two fully connected layers and a GELU activation function, which incorporates conditional features. F Mapped to intermediate representation: ,in, W 1, W 2 represents the learnable weight matrix. b 1, b 2 is the bias parameter.

[0067] (b) Spatial Attention Mechanism: A convolution-based spatial attention module (3×3 or 7×7 kernel size) is used to apply spatial dimension attention modulation to the intermediate representation, enabling the model to focus on spatially significant locations with distortion. ,in This is a temperature coefficient used to control the dynamic range of scale parameters.

[0068] (c) Zero initialization strategy: The weights of the last layer of the MLP are initialized to zero, so that in the early stages of training... c ≈ 1, β ≈0, at which point ACL is approximately an identity transformation, which is beneficial for stable training.

[0069] In ACL, given input features scaling factor and offset factor The forward affine transformation is defined as:

[0070] in This represents element-wise multiplication. The corresponding inverse transformation is:

[0071] in This represents element-wise division. This design guarantees the reversibility of the transformation—given a reverse input… It can accurately restore Meanwhile, it allows the conditional features to adaptively modulate the residual transformation process in each flow layer, thereby achieving fine-grained spatial adaptive compensation.

[0072] Step 3.3, Design of reversible linear transformation sublayer.

[0073] Each transformation layer first applies an invertible linear transformation (orthogonal linear layer) to the input features. h Perform linear mixing along the channel dimension: h' = · h + b in It is an invertible weight matrix (initialized as an orthogonal matrix to ensure invertibility). b For bias vectors, D For input feature dimensions ( D = × P × P ,in Input the number of channels. P (where the block size is). The logarithmic determinant of this transformation is:

[0074] The role of reversible linear transformation is to facilitate information exchange between feature channels and enhance the expressive power of normalized flow.

[0075] Example 5 Unlike Example 4, in Step 4 of the normalized flow-based compressed image enhancement method for machine vision tasks in Example 5, the loss function is designed as follows: A multi-task joint loss function is used to train the normalized flow model and the conditional feature encoding module. The total loss function consists of three parts: negative log-likelihood loss, pixel-level detection loss, and detection consistency loss.

[0076] Step 4.1, design the negative log-likelihood loss. L NLL ); During the training phase, the compression distortion residual is first calculated: the original image... x With decoded images Perform interpolation to obtain the distortion difference. r = x - This refers to the "target compensation residual" (i.e., the effect that compensation should produce under ideal conditions); the target residual r according to P × P The block size is expanded into feature vectors, fed into a normalized stream to perform a forward transformation, and the negative log-likelihood loss is calculated:

[0077] The role of negative log-likelihood loss is to train the normalized flow model to learn the conditional probability distribution of the compressed distortion residual under given conditional information, and to normalize the complex distortion distribution to a standard Gaussian distribution. Step 4.2, design the pixel-level detection loss. L PIXEL ); During the inference phase, latent variables are sampled from a standard Gaussian distribution. Compensation residuals are generated through inverse reversible transformation. It is then superimposed pixel by pixel onto the decoded image. To obtain an enhanced image = + The enhanced image is fed into the SOD network to obtain the enhanced saliency probability map. = Sigmoid(SOD( )); Calculate the augmented probability map and the true label. maskL1 loss between:

[0078] This loss is optional (the weight can be set to 0) and is used to provide pixel-level fidelity supervision.

[0079] Step 4.3, Design the Detection Consistency Loss (SOD Task Loss). L SOD ); The logarithmic graph of the output of the enhanced SOD network With real labels mask Calculate the combined SOD loss between:

[0080] The definitions of each item are as follows: Weighted binary cross-entropy loss An edge-aware weighted strategy is employed, assigning higher weights to salient target edge regions, making the model more focused on compensation accuracy at boundaries; the weight map is calculated from the local average pooling difference of the true labels. The weighted BCE loss is:

[0081] Weighted intersection and combination loss :

[0082] loss( L F-measure ):

[0083] in TP = Σ( prediction ⊙ mask ), FN = Σ( mask ) - TP , FP = Σ( prediction ) - TP , (Classic settings, strong accuracy).

[0084] Step 4.4, Design the total loss function; The above losses are summed according to preset weights to obtain the total training loss:

[0085] in , , The preset weighting coefficients are used; the preferred configuration is as follows: = 5×10 -4 , = 50, = 0.4; The total loss is dominated by the task loss, while the NLL loss plays an auxiliary regularization role, guiding the latent space to follow a Gaussian distribution.

[0086] Example 6 Unlike Example 5, in Step 5 of the normalized flow-based compressed image enhancement method for machine vision tasks in Example 6, the loss function is specifically designed as follows: Step 5.1, Data Preparation: Sample raw images from the training set. x and corresponding pixel-level saliency annotations mask ;Will x The image is obtained by compressing it using a compression model and then decoding it. and decoding side potential features ; Step 5.2, Conditional Feature Extraction: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Input the frozen SOD network to obtain the SOD logarithmic graph (SOD). SOD ( Decoding-side latent features Upsampling features and Channels are concatenated and encoded by the conditional feature coding module (SG-CFE) to obtain conditional features. F ; Step 5.3, Target Residual Calculation: Calculate the original image... x With decoded images Perform the difference calculation; calculate the target residual. r = x - .Will r according to P × P block size ( P =4) Expand as D 3D feature vector; Step 5.4, Forward Transformation and NLL Calculation: Calculate the target residual. r Input the normalized flow model Φ, with conditional features F To guide execution L Layer forward reversible transformation yields latent variables z = Φ( r | F ); Calculate the negative log-likelihood loss using the formula. L NLL ; Step 5.5, Inverse Transformation and Task Loss Calculation: From Standard Gaussian Distribution Randomly sampled latent variables in (0, I) z sample Compensation residuals are generated through inverse reversible transformation. The compensated residual is superimposed pixel by pixel onto the decoded image. The enhanced image is then input into the SOD network to obtain: SOD en = SOD( + ),calculate L SOD loss; Step 5.6, Parameter Update: Based on total loss To optimize the objective, update the normalized flow model parameters θ. flow and conditional feature encoding module parameter θ encoder The parameters of the compressed model and the SOD model remain frozen and do not participate in gradient updates. Step 5.7, Iterative training: Repeat steps 5.1 to 5.6 until the preset number of training epochs is reached or the early stopping condition is triggered.

[0087] Example 7 Unlike Example 6, in Example 7, step 6 of the normalized flow-based compressed image enhancement method for machine vision tasks of the present invention specifically includes: Step 6.1, Input the compressed / decoded image to be processed. Extracting conditional features F ; Step 6.2, from the standard Gaussian distribution Randomly sampled latent variables in (0, I) ; Step 6.3: Generate the compensation residual through the inverse reversible transformation of the normalized flow: ; Step 6.4, compensate for the residual The compensated features are obtained by superimposing pixel-by-pixel onto the decoded image; in the SOD task scenario, the compensated SOD response is: SOD en = SOD( + ) ; Step 6.5: Feed the compensated features into the downstream task model (such as the SOD network) to obtain the final prediction result.

[0088] Multiple random samplings can be performed. K (times), generate K Different compensation results are used to further improve performance and robustness through ensemble averaging. In fact, experiments show that a single sampling ( K =1) Significant and stable performance improvements have been achieved.

[0089] Conditional temperature control: Temperature parameters can be introduced during the reverse transformation. t Adjust the modulation intensity of the conditional information. Temperature t ∈ [0, 1], t When =0, the conditional information has no effect (i.e., it is generated without conditions). t When =1, the conditional information is fully modulated. This can be achieved by adjusting... t This allows for a balance between compensation intensity and image fidelity. Preferred t =0.8.

[0090] Application examples This invention enhances the salient target detection of remote sensing images based on DCAE compression, specifically as follows: Step A: System preparation.

[0091] Prepare the DCAE compression model (pre-trained on the DF2K general image dataset, compression parameter λ=0.0018 corresponds to approximately 0.048 bpp), and pre-train the SOD model (FreMaNet, pre-trained on the EORSSD training set, parameters frozen). The normalized flow model uses... L =12-layer ACL, block size P =4, conditional feature dimension 128, hidden layer dimension 256. The MLP in the adaptive affine module is a two-layer fully connected layer (input 256-dimensional → 512-dimensional → 256-dimensional → 48-dimensional, spatial attention convolution kernel size is 3×3, temperature coefficient... t =0.8.

[0092] Step B: Training Data. The training sets of the three datasets—ORSSD (600 images), EORSSD (1400 images), and ORSI-4199 (2000 images)—were merged, resulting in a total of 4000 remote sensing images. All images were scaled to 384×384 resolution. Online data augmentation was performed using random horizontal and vertical flipping.

[0093] Step C: Training. The Adam optimizer is used with an initial learning rate of 1×10⁻⁶. -5 Multi-step decay (decay to 1×10 in the 1200th round) -6 The batch size is 2, and the training run is 2000 epochs. The loss is configured for pure NLL training mode. l 1 = 1.0, l 2=0, l (3=0), the S-measure is evaluated once per round on the validation set, and the early stopping patience value is 5 rounds. Training is performed using a single NVIDIA RTX 3090 (24GB VRAM).

[0094] Step D: Inference and Evaluation. After training, the normalized flow model is loaded onto the decoder. For each test image, an inverse transformation (temperature) is performed. t =0.8), taking approximately 23 ms (384×384 image, RTX 3090). Nine SOD metrics were independently evaluated on three test sets. The results show that at all bitrates ( l SOD performance showed consistent improvement across all datasets (0.0018 / 0.0035 / 0.0067 / 0.013, ...

[0095] Step E: Cross-compression architecture deployment. The normalized streaming model trained on DCAE (without any fine-tuning) will be directly deployed to the TCM compressed model backend. On the EORSSD test set, TCM ( l The MAE of (=0.0025) decreased from 0.0101 to 0.0080 (a decrease of 20.8%), and the S-measure increased from 0.9082 to 0.9161 (+0.9%), verifying the plug-and-play cross-architecture generalization capability.

[0096] Scratch Training: Based on the application example, the loss configuration is changed to pure NLL training. l 1 = 1.0, l 2=0, l (3=0), without relying on any SOD supervised loss. This setting verifies that an effective distortion compensation distribution can be learned simply by maximizing the log-likelihood, without needing to calculate the task loss during the training phase, further demonstrating the simplicity and effectiveness of the normalized flow method. Experimental results show that in the pure NLL training mode, the model can still generate high-quality compensation residuals during inference, and its performance is comparable to that of the joint loss training mode.

[0097] Example of temperature parameter adjustment: During the inference phase, the temperature parameter is adjusted... t Conduct adjustment experiments. t Scan from 0.0 to 1.0 (step size 0.2). The results show that... t The performance improvement is minimal but still positive when τ = 0.0 (unconditional generation); as τ increases, the performance gradually improves. t The optimal value is reached at 0.8. t Performance at =1.0 and t =0.8 remains basically unchanged or decreases slightly. This indicates that moderate temperature control (0.8) can effectively suppress random noise introduced by latent space sampling while maintaining sufficient compensation strength.

Claims

1. A normalized flow-based compressed image enhancement method for machine vision tasks, characterized in that, The specific steps are as follows: Step 1: Obtain latent features and decoded image on the decoding side; Step 2: Obtain task prior features by taking the decoded image as input, and perform feature fusion and extraction on the task prior features, decoding-side latent features and decoded image to obtain conditional features; Step 3: Based on the conditional features, construct a conditional normalized flow model; Step 4: Design the loss function; Step 5: Use the loss function to train the conditionally normalized flow model constructed in Step 3 and update the model parameters. Step 6: Use the conditional normalized flow model obtained in Step 5 to perform inference testing.

2. The compressed image enhancement method based on normalized flow for machine vision tasks according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1, extract the original image Input compression encoder Ecomp( The bitstream is generated after quantization and arithmetic encoder. Step 1.2: Process the bitstream obtained in Step 1.1 using an arithmetic decoder to obtain the latent features on the decoding side. , dimension M × ( H / 16) × ( W / 16), of which M The number of potential channels in the compression model; Step 1.3, at the decoding end, latent features on the decoding side. via decoder Dcomp( Reconstruct to obtain the decoded image. =Dcomp( ).

3. The compressed image enhancement method based on normalized flow for machine vision tasks according to claim 1, characterized in that, Step 2 specifically involves: decoding the 3-channel image. Decoding-side latent features of channel 1 upsampling features With 1-channel task prior feature SOD ( By concatenating the features along the channel dimension, a 5-channel combined feature is obtained. , SOD ( Afterwards, the conditional features are encoded by a convolutional network in the conditional feature encoding module to output the conditional features. ,in, This is the output feature dimension.

4. The compressed image enhancement method based on normalized flow for machine vision tasks according to claim 3, characterized in that, The specific steps for constructing the conditionally normalized flow model in step 3 are as follows: Step 3.1, the normalized flow model Φ is derived from... L It is formed by layer transformation and stacking. L For each layer, the integer is a positive integer greater than or equal to 2. Each layer consists of two sub-layers connected in series: an "affine coupling layer (ACL)" and a "reversible linear transformation". The overall forward transformation is represented as: in r For input features, F The conditional features output by the conditional feature encoding module, z being a latent variable, are transformed inversely as follows: in The generated compensation residual is used to subsequently overlay the image with the decoded image to obtain the compensated image; During the training phase, a forward transformation is used to transform the input target residual. r The latent variable z is mapped and its distribution is approximated to a standard Gaussian distribution by maximizing the likelihood constraint; in the inference phase, an inverse transformation is used to sample the latent variable from the standard Gaussian distribution. The inverse transformation generates the compensation residual. To compensate for the impact of compression distortion on the performance of downstream tasks; Step 3.2, each ACL will share the same conditional features. F Mapped to a scaling factor specific to this layer. γ and offset factor β It consists of the following components: Multilayer perceptron: Consists of two fully connected layers and a GELU activation function, which integrates conditional features. F Mapped to intermediate representation: ,in, W 1, W 2 is the learnable weight matrix. b 1, b 2 represents the bias parameter; Spatial attention mechanism: A convolution-based spatial attention module is used to apply spatial dimension attention modulation to the intermediate representation, enabling the model to focus on spatially significant locations with distortion. ,in This is a temperature coefficient used to control the dynamic range of scale parameters; Zero initialization strategy: The weights of the last layer of the MLP are initialized to zero, so that in the early stages of training... γ ≈ 1, β ≈ 0, at which point ACL is approximately an identity transformation, which is beneficial for stable training; In ACL, given input features scaling factor and offset factor The forward affine transformation is defined as: in To represent element-wise multiplication, the corresponding inverse transformation is: in This represents element-wise division; the design ensures the reversibility of the transformation—given a reverse input. It can accurately restore Meanwhile, it allows the conditional features to adaptively modulate the residual transformation process in each flow layer, thereby achieving fine-grained spatial adaptive compensation. Step 3.3: For reversible linear transformation layers, each transformation first applies a reversible linear transformation to the input features. h Perform linear mixing along the channel dimension: h' = · h + b in It is an invertible weight matrix. b For bias vectors, D For the input feature dimension, D = × P × P ,in Input the number of channels. P Let the block size be denoted by ; the logarithmic determinant of this transformation is: The role of reversible linear transformation is to facilitate information exchange between feature channels and enhance the expressive power of normalized flow.

5. The compressed image enhancement method based on normalized flow for machine vision tasks according to claim 4, characterized in that, The loss function designed in step 4 is as follows: Step 4.1, design the negative log-likelihood loss; During the training phase, the compression distortion residual is first calculated: the original image... x With decoded images Perform interpolation to obtain the distortion difference. r = x - This is referred to as "target compensation residual"; the target residual... r according to P × P The block size is expanded into feature vectors, fed into a normalized stream to perform a forward transformation, and the negative log-likelihood loss is calculated: The role of negative log-likelihood loss is to train the normalized flow model to learn the conditional probability distribution of the compressed distortion residual under given conditional information, and to normalize the complex distortion distribution to a standard Gaussian distribution. Step 4.2, design pixel-level detection loss; During the inference phase, latent variables are sampled from a standard Gaussian distribution. Compensation residuals are generated through inverse reversible transformation. It is then superimposed pixel by pixel onto the decoded image. To obtain an enhanced image = + The enhanced image is fed into the SOD network to obtain the enhanced saliency probability map. = Sigmoid(SOD( )); Calculate the augmented probability map and the true label. mask L1 loss between: This loss is optional and is used to provide pixel-level fidelity supervision; Step 4.3, design the consistency loss for detection; The logarithmic graph of the output of the enhanced SOD network With real labels mask Calculate the combined SOD loss between: The definitions of each item are as follows: Weighted binary cross-entropy loss The edge-aware weighting strategy is adopted to give higher weights to the edge regions of significant targets, so that the model pays more attention to the compensation accuracy at the boundary. The weighted graph is calculated from the local average pooling difference of the true labels: The weighted BCE loss is: Weighted intersection and combination loss : Loss (L) F-measure ): among them TP = Σ( prediction ⊙ mask ), FN = Σ( mask ) - TP , FP = Σ( prediction )- TP , ; Step 4.4, Design the total loss function; The above losses are summed according to preset weights to obtain the total training loss: in , , The preset weighting coefficients are used; the preferred configuration is as follows: = 5×10^ -4 , = 50, = 0.4; The total loss is dominated by the task loss, while the NLL loss plays an auxiliary regularization role, guiding the latent space to follow a Gaussian distribution.

6. The compressed image enhancement method based on normalized flow for machine vision tasks according to claim 5, characterized in that, Step 5 specifically involves: Step 5.1, Data Preparation: Sample raw images from the training set. x and corresponding pixel-level saliency annotations mask ; Will x The image is obtained by compressing it using a compression model and then decoding it. and decoding side potential features ; Step 5.2, Conditional Feature Extraction: [The text abruptly ends here, likely due to an incomplete sentence or Input the frozen SOD network to obtain the SOD logarithmic graph (SOD). SOD ( Decoding-side latent features Upsampling features and Channels are concatenated and encoded by a conditional feature encoding module to obtain conditional features. F ; Step 5.3, Target Residual Calculation: Calculate the original image... x With decoded images Perform the difference calculation; calculate the target residual. r = x - ; Will r according to P × P The block size is expanded to D 3D feature vector; Step 5.4, Forward Transformation and NLL Calculation: Transform the target residual r Input the normalized flow model Φ, with conditional features F To guide execution L A forward invertible transformation of the layer yields the latent variable z = Φ( r | F ); Calculate the negative log-likelihood loss using the formula. L NLL ; Step 5.5, Inverse Transformation and Task Loss Calculation: From Standard Gaussian Distribution Randomly sampled latent variables in (0, I) z sample Compensation residuals are generated through inverse reversible transformation. The compensated residual is superimposed pixel by pixel onto the decoded image. The enhanced image is then input into the SOD network to obtain: SOD en = SOD( + ),calculate L SOD loss; Step 5.6, Parameter Update: Based on total loss To optimize the objective, update the normalized flow model parameters θ. flow and conditional feature encoding module parameter θ encoder The parameters of the compressed model and the SOD model remain frozen and do not participate in gradient updates. Step 5.7, Iterative training: Repeat steps 5.1 to 5.6 until the preset number of training rounds is reached or the early stopping condition is triggered.

7. The compressed image enhancement method based on normalized flow for machine vision tasks according to claim 6, characterized in that, Step 6 specifically includes: Step 6.1, Input the compressed / decoded image to be processed. Extracting conditional features F ; Step 6.2, from the standard Gaussian distribution Randomly sampled latent variables in (0, I) ; Step 6.3: Generate the compensation residual through the inverse reversible transformation of the normalized flow: ; Step 6.4, compensate for the residual The compensated features are obtained by superimposing pixel-by-pixel onto the decoded image. In the SOD task scenario, the compensated SOD response is: SOD en = SOD( + ) ; Step 6.5: Feed the compensated features into the downstream task model to obtain the final prediction result.

8. The compressed image enhancement method based on normalized flow for machine vision tasks according to claim 7, characterized in that, In step 6: multiple random samplings can be performed to generate... K Different compensation results are obtained, and performance and robustness are further improved by integrating and averaging.

Images