Diffusion image compression and reconstruction method combining semantic guidance and regional detail enhancement

Abstract

The invention discloses a diffusion image compression and reconstruction method combining semantic guidance and regional detail enhancement, and the method comprises the steps: extracting image semantic features through a pre-training visual semantic model, carrying out the PCA dimension reduction, generating semantic prior embedding matched with a diffusion potential space, and meanwhile, generating a structure perception ROI mask corresponding to small contents in the potential space; in the diffusion forward coding process, the KL divergence of each time step is adaptively divided between the ROI and the background, and two paths of noise are respectively written in through a Gaussian channel; in a reverse denoising stage, a condition guidance weight of spatial variation is constructed based on semantic prior and ROI mask, and pixel-by-pixel fusion is performed on noise prediction of unconditional branches and conditional branches, so that the definition and the stability of fine structures such as characters and human faces can be remarkably improved under the condition of low bit rate, and the method is suitable for large-scale popularization and application. Meanwhile, the naturalness of the overall structure and texture is kept, and the method has high detail fidelity, good subjective and objective rate distortion performance and high practical popularization value.

Description

Technical Field

[0001] This invention relates to the field of generative image compression technology, specifically to a diffusion image compression and reconstruction method that combines semantic guidance and region detail enhancement. Background Technology

[0002] Traditional lossy image compression methods, such as JPEG, JPEG2000, and transform-based standards like HEVC and VVC, can achieve good subjective quality at medium to high bitrates through fixed transform bases and entropy coding mechanisms. However, these methods typically employ hand-designed rate-distortion models and uniform quantization strategies, treating different semantic regions in the image equally. They lack explicit resource allocation for detailed areas such as small fonts, fine lines, and facial features, leading to severe loss of high-frequency textures and structural details at low bitrates.

[0003] In recent years, deep learning-driven end-to-end compression methods (such as learned image compression based on autoencoders) have significantly improved rate-distortion performance under a given bit budget by learning nonlinear transformations and probabilistic models. However, most of these methods rely on globally uniform distortion metrics (such as MSE) and uniform latent space quantization, making it difficult to differentiate different regions based on downstream perceptual tasks or semantic importance.

[0004] Currently, generative compression methods have shown advantages in extremely low bitrate scenarios. Compression methods based on Generative Adversarial Networks (GANs) learn the prior distribution of images to generate visually natural and texture-rich reconstruction results at low bitrates. Compression methods based on Diffusion Models utilize Markov chains with forward noise addition and backward denoising to map the original image to a noise space, write a small amount of information through a constrained Gaussian channel, and then reconstruct the image using a powerful generative model prior.

[0005] However, existing generative compression methods still suffer from the following problems: Lack of structure-aware bit allocation mechanisms: Most methods only control the overall bitrate, failing to distinguish between "semantically critical regions" and "background regions" during encoding. They cannot allocate limited bits to detail-sensitive regions such as text and faces, leading to severe blurring or artifacts in these areas at low bitrates. Inconsistent spatial control of guidance intensity in the denoising stage: The classifier-free guidance in diffusion models typically uses a single scalar intensity, treating the entire image equally. This fails to apply stronger semantic constraints to local regions during denoising inversion and weaker guidance to background regions, resulting in global consistency but insufficient local detail. Lack of local detail enhancement mechanisms deeply coupled with the diffusion process: Some works attempt to sharpen or enlarge local regions after reconstruction, but these often remain at a simple post-processing level, without dedicated reconstruction of local regions along the encoding and denoising trajectories. This makes it difficult to fully utilize the generative capabilities of diffusion models for targeted enhancement of subtle details.

[0006] Therefore, how to introduce structure-aware bit allocation strategies and spatially adaptive denoising guidance mechanisms within the diffusion compression framework, and combine local magnification reconstruction with soft fusion to achieve high-quality reconstruction of small content regions while maintaining overall bitrate control, has become an important issue in current research. Summary of the Invention

[0007] Purpose of the invention: The purpose of this invention is to address the shortcomings of existing technologies and provide a diffusion image compression and reconstruction method that combines semantic guidance and regional detail enhancement.

[0008] Technical solution: The present invention provides a diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement, comprising the following steps:

[0009] Step 1: Semantic prior and latent space representation, i.e., for the input image Extract text features or image semantic features, and use a pre-trained vision-language model to obtain high-dimensional semantic embeddings. And obtain low-dimensional semantic vectors through linear dimensionality reduction. , to transform low-dimensional semantic vectors It is mapped to a conditional bias tensor B consistent with the latent space channel, which is used to adjust the generation distribution during subsequent diffusion.

[0010] Simultaneously input image Inputting a variational autoencoder yields the initial latent variables that serve as the compression target. (No diffused noise added);

[0011] Step 2: Semantic and Region-Aware Encoding

[0012] First, a Region of Interest (ROI) mask is constructed based on semantic information or external annotations. The current noise state latent variable (the current noise state latent variable refers to the latent space state participating in coding at the current moment in the coding stage, which evolves gradually from the initial latent variable through several steps of diffusion and noise addition and channel coding) is flattened into a one-dimensional vector and divided into a set of ROI indexes and a set of background indexes (ROI sub-vectors and background sub-vectors). Then, given the total KL divergence constraint at each step, the KL quotas of the ROI and background regions are calculated using time-related weight coefficients, and the total KL is proportionally divided into ROI KL and background KL.

[0013] The region of interest subvector and the background subvector are encoded separately using a Gaussian channel simulator. After recombination, the noise latent variable is updated according to the forward diffusion formula to obtain the updated noise latent variable. This allows for bit tilting of the region of interest while maintaining overall bit rate control. Here, the noise latent variable refers to the latent variable that is gradually noise-added and updated over time during the diffusion and Gaussian channel coding process. In short, the initial latent variable becomes the final noise latent variable after multiple noise-addition and coding steps.

[0014] Step 3: Denoising and Reconstruction Guided by Spatial Semantics

[0015] At the decoding end, the terminal noise state latent variable is recovered based on the recorded channel state. A spatial guidance map determined by the semantic mask and high-frequency response is constructed. Unconditional prediction noise and conditional prediction noise are calculated at each reverse time step, and then... The noise estimate is obtained by weighting the data pixel by pixel in the form of spatial adaptive guidance. The latent variables are iteratively updated along the diffusion inversion trajectory to obtain the denoised latent space representation. The full-image basic reconstruction image is then recovered by the decoder.

[0016] Step 4: Zoom in and enhance details of the region of interest.

[0017] First, the local ROI magnification and soft fusion module crops one or more regions of interest on the full-image reconstruction based on predefined coordinates or annotation files. After the cropped block is magnified according to the target resolution, the semantic prior construction and encoding and denoising processes are repeated to obtain a high-quality local reconstruction result.

[0018] For each region of interest, a soft-fusion mask is generated with a center weight of 1 and an edge weight that gradually decays to 0. The local high-quality reconstruction result is then weighted and fused with the initial global reconstruction result within the corresponding region to obtain...

[0019] ;

[0020] This improves the clarity and structural consistency of reconstructed small content areas such as text and faces under a unified bitrate constraint.

[0021] Furthermore, the detailed method of step 1 is as follows:

[0022] Step 1.1: For the input image as Feature extraction is performed using a pre-trained vision-language model to obtain a high-dimensional semantic embedding vector with a dimension of 512. The loaded principal component analysis parameter matrix and mean vector are subjected to linear transformation PCA to obtain a dimension of low-dimensional semantic vectors Then, using a fixed linear projection matrix, The conditional bias tensor B, which is consistent with the number of latent space channels, is mapped to the target distribution q through guiding coefficients during the diffusion coding stage to adjust the subsequent noise sampling distribution.

[0023] The PCA transformation of this invention is uniquely determined by the mean vector and the projection matrix. Only one linear transformation is required during the online inference stage, resulting in low computational overhead. Subsequently, s is mapped to a conditional bias vector consistent with the number of latent space channels through a fixed linear projection matrix, and expanded into a bias tensor B that matches the shape of the latent space, serving as an additional offset to the target Gaussian coding distribution. Here, the dimensionality-reduced semantic vector is mapped to a conditional bias consistent with the number of latent space channels and is explicitly used as the "semantic modulation quantity" in the Gaussian channel / diffusion process.

[0024] Step 1.2, for the input image as First, the initial latent variables are encoded using a variational autoencoder that matches the diffusion model. , as the target representation of compression;

[0025] In the forward diffusion process, the initial latent variables According to noise scheduling sequence The noise is gradually increased, and a joint distribution is used to characterize it, as shown in the following expression:

[0026] ;

[0027] in, The cumulative coefficient corresponding to the noise scheduling sequence;

[0028] During the encoding phase, when the time step After falling within the preset injection ratio, a set of time-varying guiding coefficients are used. The conditional bias tensor B is additively injected into the normalized target vector, i.e., the calculation is performed. : ;

[0029] in , The prior mean and standard deviation of the forward diffusion process at the current time step. Based on target latent variables and current noise state The posterior mean obtained through analysis;

[0030] Then updated to: ;

[0031] In the above formula, This refers to the semantic guidance coefficient at the k-th time step. This is used to control the intensity of injecting the conditional bias tensor B into the normalized target vector at this time step. It is calculated based on the normalized position of the current time step on the entire diffusion time axis using a preset ramp function. The closer to the later time step, the stronger the slope. The larger the value, the more emphasis is placed on the influence of semantic priors on the target distribution in the later stages;

[0032] The diffusion model learns the inverse conditional distribution. Predictive noise is typically used. Given in the form of:

[0033] ;

[0034] in , , The signal-to-noise ratio (SNR) is calculated analytically from the current and previous time steps. Based on this, the present invention treats each sampling process as Gaussian channel coding and introduces semantic prior and structure-aware mechanisms to achieve bit tilting and detail enhancement for the region of interest (ROI).

[0035] Through the aforementioned semantic bias injection, the encoding stage, without altering the diffusion model structure, provides a stronger consistency constraint on the overall semantics of the image during the Gaussian channel coding process. Let qn(k) represent the normalized target vector obtained at the k-th time step. Its function is to unify the "target distribution to be approximated" in the current step to the same scale and use it as the input target for Gaussian channel coding. Subsequent updates and operations, whether adding semantic bias B or performing KL allocation for ROI / background, are based on this normalized target vector qn(k).

[0036] Furthermore, the method for masking the Region of Interest (ROI) in step 2 is as follows:

[0037] First, the input mask Adjust the dimensions (H, W) to the same height and width as the latent variable space using nearest neighbor interpolation, if a mask is input. For a single channel, it is expanded to (B, 1, H, W) in both the batch and channel dimensions, and then copied along the channel dimension to (B, C, H, W); subsequently, the mask is flattened into a one-dimensional vector and thresholded. Binarization is performed to obtain the Boolean index vector. The remaining positions serve as background indexes. Where B represents the batch size, i.e. the number of images (or latent variables) fed into the encoder at one time; C represents the number of channels, which is consistent with the channel dimension of the current latent variable; H and W represent the height and width of the latent variable space, respectively, which are the resolution in the spatial dimension after VAE encoding.

[0038] When Boolean index vector When the proportion of true values ​​in the image is too small or close to 1, determine the current image (i.e., the original input image). Without a valid Region of Interest (ROI) mask, it degenerates into a global single-channel encoding.

[0039] After obtaining the Region of Interest (ROI) mask, continue with the following steps:

[0040] At each time step k, the flattened target distribution vector By index Divided into sub-vectors of the region of interest With background subvectors And based on the normalized time ratio With preset start / stop ratio Calculate slope weight Used to generate the ROI weight coefficients for the current step. With background weight coefficient The calculation formula is as follows:

[0041] ;

[0042] ;

[0043] In the above formula, This represents the set of all element locations belonging to the Region of Interest (ROI), used to index and obtain the ROI subvector; This represents the set of remaining background locations, used to index and obtain the background subvector. Subsequent assignment of KL to the ROI and background, and the application of Gaussian channel coding, are all accomplished through these two indices.

[0044] It is based on the normalized time ratio and start / stop intervals The calculated slope weight is made to change monotonically with time using a cosine function and smooth amplification. The KL weighting factor assigned to the ROI region at that time step is... and Two hyperparameters passed Obtained through linear interpolation; These are the KL weighting factors assigned to the background region, by , and Linear interpolation yields KL weighting coefficients for the ROI path and background path in the code, which are used to dynamically tilt more bits to the ROI region while keeping the total KL constraint unchanged.

[0045] Let the number of regions of interest and the number of background elements be respectively... , Then construct

[0046] ;

[0047] ;

[0048] and This represents the weighted total of the ROI region and the background region at the k-th time step. It is obtained by multiplying the corresponding time weight coefficient and the number of elements, and is used to measure how much attention should be paid to the two regions at this time step.

[0049] Then calculate the total KL budget The formula is:

[0050] ;

[0051] in, This is the total KL budget for this time step. and It is based on the former and is proportionally split and allocated to the ROI and background KL constraints to determine how many bits each block can use;

[0052] The region of interest (ROI) is split into a region of interest (KL) and a background (KL). Then, the ROI and background KL are respectively fed into a Gaussian channel simulator for encoding. , Recombined into a complete sample and based on

[0053] ;

[0054] Update the noise latent variables to achieve dynamic bit tilting of the region of interest;

[0055] By repeating the above process over the entire time step sequence, while ensuring that the global bit rate remains unchanged, time-related bit tilt is applied to the ROI region, so that the detail region has a more stable channel state and stronger reconstructability in the noise space.

[0056] Furthermore, step 3 introduces semantic priors and spatial structure information for encoding and denoising reconstruction. Spatial structure information refers to the spatial distribution prior obtained from semantic annotation or external information. Specifically, it is manifested as ROI masks and their smoothed, downsampled, and upsampled forms, describing which locations in the image belong to small content regions such as text and faces, and the shape, extent, and relative positional relationships of these regions, rather than simply a numerical scalar. In subsequent encoding and denoising stages, this spatial structure information is further processed to generate a spatial guidance map that actually participates in modulation noise prediction and KL allocation. The specific reconstruction method is as follows:

[0057] First, at each backward time step, the normalization ratio r is calculated based on the current step index and the total number of steps, within the interval... Constructing the ramp coefficient using the cosine function ,

[0058] ;

[0059] This represents the time weighting coefficient obtained based on the cosine curve. It maps the normalized time progress τ to the range of 0–1 and is used to control the overall strength of semantics and high-frequency guidance on different backsteps.

[0060] Secondly, in the same noise state Predicting noise under the following computational conditions With unconditional prediction noise residual difference of paper Then, average along the channel dimension and normalize to [0,1] in the spatial dimension to obtain the high-frequency saliency map. This is used to reflect the strength of the influence of semantic guidance on different spatial locations in the current step.

[0061] ;

[0062] ;

[0063] It is the basic guiding strength of the spatial guidance diagram Before considering late-stage amplification, the initial proportion of conditional and unconditional branches for each spatial location is determined by a weighted fusion of the ramp, semantic mask, and high-frequency graph. This is a weighted map obtained by normalizing the high-frequency saliency map (derived from the difference between conditional and unconditional noise) to the 0–1 interval. It is used to highlight detailed regions with significant high-frequency differences. These are high-frequency weighting coefficients that control the high-frequency saliency plot. The higher the value of the influence weight in the middle, the stronger the contribution of high-frequency differences to the final spatial guidance;

[0064] Next, the smoothed semantic mask (soft semantic graph) is applied. With preset gate value according to By fusing semantics and high-frequency joint weights, a basic guidance graph is obtained. :

[0065] ;

[0066] This refers to the effective semantic graph; here it is the smoothed semantic mask. This refers to the soft semantic map; the original semantic mask is first processed by interpolation, convolution, or Gaussian filtering to perform spatial smoothing and normalization, transforming the original 0 / 1 hard mask into a weighted map with continuously taking values. This continuous weighted form is referred to as the "soft semantic map" in the paper. It is then fused with gating values ​​and further combined with high-frequency significant values. Figure 1 This forms the basic guide diagram;

[0067] When r > 0.7, for Multiply by the subsequent enhancement factor Finally, cut to Spatial guidance coefficient diagram obtained from the interval ;

[0068] And perform a pixel-wise linear combination of unconditional and conditional noise predictions, to The original scalar CFG form is replaced in the diffusion inversion update, as shown in the following formula: ;

[0069] Perform pixel-by-pixel weighted combination, Noise estimates are used as part of the diffusion inversion step to update latent variables.

[0070] Here, a spatial guidance map is constructed by combining semantic masking and high-frequency response for pixel-by-pixel CFG. Local ROI magnification and reconstruction are introduced and seamlessly integrated with the full image result using a soft fusion mask to improve the reconstruction quality of small areas. Strong guidance is achieved in the ROI area and weak guidance in the background area on the denoising trajectory, taking into account both detail enhancement and overall naturalness.

[0071] Step 3 above completes a denoising reconstruction guided by spatial semantics on the whole image, resulting in a global "whole image base reconstruction image". It only outputs the reconstructed image and the corresponding latent variable representation, and does not generate new bounding boxes.

[0072] Furthermore, the detailed method of step 4 is as follows:

[0073] For each region of interest bounding box First, calculate the magnification based on its width and height.

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] The above region of interest box The coordinates or annotation files are derived from a predefined coordinate or annotation file. Then, the preset region of interest bounding box is projected onto the full-image base reconstruction image obtained in step 3, and local cropping, magnification and detail enhancement are performed. These are the x and y coordinates of the top-left pixel of the region of interest bounding box; These are the x and y coordinates of the bottom right corner pixel of the region of interest; these four numbers together determine the start and end positions of the local area to be cropped and enlarged within the entire image; the aforementioned " , , , " is an intermediate value used to determine the final magnification (s), and w and h are the current region of interest boxes. The width and height on the image are used to characterize the original size of this local block. and It is based on the preset minimum target side length With the maximum target side length , combined and The derived lower and upper bounds of the magnification are: Ensure that the shortest side reaches at least after enlargement. (And not less than 1, it will not shrink), Ensure that the longest side does not exceed [a certain value] after enlargement. ;

[0079] Let the magnification factor be... The cropped image is enlarged by bicubic interpolation; the cropped image here refers to the local image patch that is cropped out according to the region of interest box b on the reconstructed image based on the whole image, and then the local image is enlarged by bicubic interpolation.

[0080] Subsequently, the low-dimensional semantic prior of the local image is recalculated, and the encoding function is called under a given encoding time step schedule to obtain the noisy reconstruction latent variables corresponding to each candidate time step. Using a denoising function with spatial semantic guidance enabled, these candidate latent variables are inverted one by one to obtain a set of locally reconstructed images. The performance of these images with the original local image in terms of peak signal-to-noise ratio (PSNR) or perceptual similarity index is compared. The time step corresponding to the reconstruction result with the highest PSNR or lowest LPIPS is selected as the optimal step size. The noisy latent variables at this time step are then denoised and inverted again to generate the final locally reconstructed image. The aforementioned original local image refers to the original input image. Within the region of interest Sub-images obtained by inner cropping.

[0081] Furthermore, in step 4, the local region of interest magnification and detail enhancement stage, a soft fusion mask is used to fuse the final local reconstructed image and the overall base reconstructed image.

[0082] First, a two-dimensional soft-blending mask M is generated based on the local bounding box size (h,w), with a center weight of 1 and gradually decreasing to 0 at the edges. The Chebyshev distance relative to the center is calculated by constructing a row and column grid (x,y). , and according to

[0083] ;

[0084] Obtain the rectangular soft-blending weights; if a low-resolution soft mask is provided externally, first bilinearly interpolate the soft mask to the local bounding box size, then multiply it point-by-point with the soft-blending mask, and finally crop it to [0,1] to obtain the final blending weights. , local patch image Reconstructed image based on full map By pressing the merge button within the corresponding area, the final reconstructed image after fusion is obtained. The expression is as follows

[0085] ;

[0086] A weighted summation is performed, and the merged pixels are written back to their original positions, thereby improving the reconstruction quality of local detail areas without introducing obvious boundary artifacts.

[0087] It is a high-quality local result obtained by enlarging and reconstructing a local patch individually. This is the obtained full-image basic reconstruction image. It is a soft fusion weight map with values ​​of 0–1 within the corresponding ROI region.

[0088] Beneficial effects: Without changing the basic diffusion model and variational autoencoder structure, this invention introduces semantic prior embedding, a structure-aware region of interest (ROI) bit allocation module, a spatially adaptive denoising and reconstruction module, and a local ROI amplification, reconstruction, and soft fusion module to achieve explicit protection and targeted enhancement of detailed regions, thereby significantly improving the reconstruction quality of small content regions under a unified bitrate constraint.

[0089] Compared with existing diffusion compression and end-to-end learned compression methods, the present invention has the following advantages:

[0090] 1. Improved efficiency of structure-aware bit allocation and bit rate utilization: This invention explicitly distinguishes between regions of interest and background regions during the encoding stage, and dynamically allocates the two types of regions in the KL budget using a time-dependent weighting function. This allows semantically critical regions to obtain more effective bits at the same total bit rate, thereby achieving a better rate-distortion tradeoff on the RD curve.

[0091] 2. Targeted reconstruction and perception quality improvement of detailed areas: The spatial semantic guidance CFG module proposed in this invention uses semantic mask and high-frequency saliency map to construct pixel-by-pixel guidance intensity, so that the diffusion inversion process applies strong constraints to areas such as text and face, and weaker constraints to background areas. Combined with the local ROI magnification reconstruction and soft fusion module, the recognizability of small font text, fine lines between lines and facial organ details is greatly improved.

[0092] 3. Compatible with existing diffusion models and low implementation cost: In terms of implementation, this invention is directly based on publicly available pre-trained diffusion models and variational autoencoders for compression and reconstruction. There is no need to retrain the diffusion network. Only by adding ROI bit allocation module, spatial CFG module and ROI amplification reconstruction and fusion module, significant detail enhancement effect can be obtained on the existing model. It has good engineering feasibility and scalability.

[0093] 4. Comprehensive Advantages in Rate-Distortion Performance and BD-Rate Metrics: This invention compares the RD curves and BD-Rate metrics of several representative traditional compression methods and generative compression methods on public datasets such as Kodak and DIV2K. Experimental results show that, in BD-Rate statistics using PSNR and LPIPS as metrics, the method of this invention has superior overall rate-distortion performance in detail-sensitive scenarios, especially in the low bit-rate region. Attached Figure Description

[0094] Figure 1 This is a flowchart illustrating the structure-aware ROI bit allocation module in this invention.

[0095] Figure 2This is a flowchart illustrating the detail reconstruction and soft fusion module in this invention;

[0096] Figure 3 is a comparison of the rate-distortion (RD) curves of the model proposed in this invention on Kodak and datasets;

[0097] Figure 4 This is a visualization comparison of the reconstruction of the model and the comparison method of this invention on the Kodak dataset;

[0098] Figure 5 This is a visualization comparison of the reconstruction of the model and the comparison method of this invention on the DIV2K dataset. Detailed Implementation

[0099] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0100] like Figure 1 As shown, the diffusion image compression and reconstruction method of the present invention, which combines semantic guidance and region detail enhancement, includes the following steps:

[0101] Step 1: Semantic prior and latent space representation, i.e., for the input image Extract text features or image semantic features, and use a pre-trained vision-language model to obtain high-dimensional semantic embeddings. And obtain low-dimensional semantic vectors through linear dimensionality reduction. , to transform low-dimensional semantic vectors It is mapped to a conditional bias tensor B consistent with the latent space channel, which is used to adjust the generation distribution during subsequent diffusion.

[0102] Simultaneously input image Inputting a variational autoencoder yields the initial latent variables that serve as the compression target. ;

[0103] Step 2: Semantic and Region-Aware Encoding

[0104] First, construct a region of interest mask based on semantic information or external annotations. After flattening the latent variables into a one-dimensional vector, divide them into a region of interest index set and a background index set.

[0105] Then, given the total KL divergence constraint at each step, the KL quotas for the region of interest and the background region are calculated using time-related weighting coefficients, and the total KL is proportionally divided into the region of interest KL and the background KL.

[0106] The region of interest subvector and the background subvector are encoded separately using a Gaussian channel simulator. After recombination, the noise latent variable is updated according to the forward diffusion formula to obtain the updated noise latent variable. ;

[0107] Step 3: Denoising and Reconstruction Guided by Spatial Semantics

[0108] Based on the latent variables of the noise state at the channel state recovery endpoint, a spatial guidance map determined by the semantic mask and the high-frequency response is constructed. Unconditional prediction noise and conditional prediction noise are calculated at each inverse time step and weighted pixel by pixel to obtain a noise estimate with spatial adaptive guidance. The latent variables are iteratively updated along the diffusion inversion trajectory to obtain the denoised latent spatial representation, and the full-image basic reconstruction image is recovered by the decoder.

[0109] Step 4: Zoom in and enhance details of the region of interest.

[0110] First, the local ROI magnification and soft fusion module crops one or more regions of interest on the full-image reconstruction based on predefined coordinates or annotation files. After the cropped block is magnified according to the target resolution, the semantic prior construction and encoding and denoising processes are repeated to obtain a high-quality local reconstruction result.

[0111] For each region of interest, a soft fusion mask is generated with a center weight of 1 and an edge weight that gradually decays to 0. The local high-quality reconstruction results are then weighted and fused with the initial reconstruction results of the whole image within the corresponding regions, thereby improving the reconstruction clarity and structural consistency of small content regions such as text and faces under a unified bitrate constraint.

[0112] This invention adds a semantic prior bias and time-scheduled KL allocation mechanism to the diffusion compression framework, enabling the originally uniform noise channel to have fine control over different regions and time steps. Based on the CFG and ROI processing methods, it no longer uses only global scalar guidance or simple pruning, but forms an adaptive enhancement method for details such as text and faces through spatially varying guidance graphs and multi-time reconstruction selection strategies.

[0113] The detailed method for step 1 in this embodiment is as follows:

[0114] Step 1.1: For the input image as Feature extraction is performed using a pre-trained vision-language model to obtain a high-dimensional semantic embedding vector with a dimension of 512. The loaded principal component analysis parameter matrix and mean vector are subjected to linear transformation PCA to obtain a dimension of low-dimensional semantic vectors Then, using a fixed linear projection matrix, The conditional bias tensor B, which is consistent with the number of latent space channels, is mapped to the target distribution q through guiding coefficients during the diffusion coding stage to adjust the subsequent noise sampling distribution.

[0115] Step 1.2, for the input image as First, the initial latent variables are encoded using a variational autoencoder that matches the diffusion model. , as the target representation of compression;

[0116] In the forward diffusion process, the initial latent variables According to noise scheduling sequence The noise is gradually increased, and a joint distribution is used to characterize it, as shown in the following expression:

[0117] ;

[0118] in, This refers to the cumulative noise parameter;

[0119] During the encoding phase, when the time step After falling within the preset injection ratio, a set of time-varying guiding coefficients are used. The conditional bias tensor B is additively injected into the normalized target vector, i.e., the calculation is performed. : ;

[0120] in , The prior mean and standard deviation of the forward diffusion process at the current time step. Based on target latent variables and current noise state The posterior mean obtained through analysis;

[0121] Then updated to: .

[0122] The method for masking the Region of Interest (ROI) in step 2 of this embodiment is as follows:

[0123] First, the input mask ( Adjust the height and width dimensions (H, W) to the same as the latent variable space using nearest neighbor interpolation. If the input mask is... If the mask is single-channel, it is expanded to (B,1,H,W) along both the batch and channel dimensions, and then copied along the channel dimension to (B,C,H,W); subsequently, the mask is flattened into a one-dimensional vector and thresholded. Binarization is performed to obtain the Boolean index vector. The remaining positions serve as background indexes. ;

[0124] When Boolean index vector When the proportion of true values ​​is too small or close to 1, it is determined that there is no effective Region of Interest (ROI) mask in the current image, and it degenerates into global single-channel coding.

[0125] After obtaining the Region of Interest (ROI) mask, continue with the following steps:

[0126] At each time step k, the flattened target distribution vector By index Divided into sub-vectors of the region of interest With background subvectors And based on the normalized time ratio With preset start / stop ratio Calculate slope weight Used to generate the ROI weight coefficients for the current step. With background weight coefficient The calculation formula is as follows:

[0127] ;

[0128] ;

[0129] Let the number of regions of interest and the number of background elements be respectively... , Then construct

[0130] ;

[0131] ;

[0132] Then calculate the total KL budget The formula is:

[0133] ;

[0134] The region of interest (ROI) is split into a region of interest (KL) and a background (KL). Then, the ROI and background KL are respectively fed into a Gaussian channel simulator for encoding. , Recombined into a complete sample and based on

[0135] ;

[0136] Update the noise latent variables to achieve dynamic bit tilting of the region of interest.

[0137] The method for reconstructing the structure by introducing semantic prior and spatial structure information in step 3 of this embodiment is as follows:

[0138] First, at each backward time step, the normalization ratio r is calculated based on the current step index and the total number of steps, within the interval... Constructing the ramp coefficient using the cosine function ,

[0139] ;

[0140] This represents the time weighting coefficient obtained based on the cosine curve. It maps the normalized time progress τ to the range of 0–1 and is used to control the overall strength of semantics and high-frequency guidance on different backsteps.

[0141] Secondly, in the same noise state Predicting noise under the following computational conditions With unconditional prediction noise residual difference of paper Then, average along the channel dimension and normalize to [0,1] in the spatial dimension to obtain the high-frequency saliency map. This is used to reflect the strength of the influence of semantic guidance on different spatial locations in the current step.

[0142] ;

[0143] ;

[0144] Next, the smoothed semantic mask (soft semantic graph) is applied. With preset gate value according to By fusing semantics and high-frequency joint weights, a basic guidance graph is obtained. :

[0145] ;

[0146] This refers to an effective semantic graph;

[0147] When r > 0.7, for Multiply by the subsequent enhancement factor Finally, cut to Spatial guidance coefficient diagram obtained from the interval ;

[0148] And perform a pixel-wise linear combination of unconditional and conditional noise predictions, to The original scalar CFG form is replaced in the diffusion inversion update, as shown in the following formula: ;

[0149] Perform pixel-by-pixel weighted combination, Noise estimation, used as a step in the diffusion inversion process, is employed to update latent variables.

[0150] The detailed method for step 4 of this embodiment is as follows:

[0151] For each region of interest bounding box First, calculate the magnification based on its width and height;

[0152] Let the magnification factor be... The cropped image is enlarged using bicubic interpolation.

[0153] Subsequently, the low-dimensional semantic prior of the local image is recalculated, and the encoding function is called under a given encoding time step schedule to obtain the noisy reconstruction latent variable corresponding to each candidate time step. The denoising function is used to invert these candidate latent variables one by one under the guidance of spatial semantics to obtain a set of local reconstructed images. The performance of these images with the original local images in terms of peak signal-to-noise ratio or perceptual similarity index is compared. The time step corresponding to the reconstruction result with the highest PSNR or the lowest LPIPS is selected as the optimal step size. The noisy latent variable under this time step is denoised and inverted again to generate the final local reconstructed image.

[0154] like Figure 2 As shown, a soft fusion mask is used to fuse the final local reconstructed image with the overall base reconstructed image:

[0155] First, a two-dimensional soft-blending mask M is generated based on the local bounding box size (h,w), with a center weight of 1 and gradually decreasing to 0 at the edges. The Chebyshev distance relative to the center is calculated by constructing a row and column grid (x,y). , and according to

[0156] ;

[0157] Obtain the rectangular soft-blending weights; if a low-resolution soft mask is provided externally, first bilinearly interpolate the soft mask to the local bounding box size, then multiply it point-by-point with the soft-blending mask, and finally crop it to [0,1] to obtain the final blending weights. , local patch image Reconstructed image based on full map Press within the corresponding area

[0158] ;

[0159] A weighted summation is performed, and the merged pixels are written back to their original positions, thereby improving the reconstruction quality of local detail areas without introducing obvious boundary artifacts.

[0160] This invention, without altering the basic diffusion model and variational autoencoder structure, introduces semantic prior embedding, a structure-aware region of interest (ROI) bit allocation module, a spatially adaptive denoising and reconstruction module, and a local ROI amplification, reconstruction, and soft fusion module to achieve explicit protection and targeted enhancement of detailed regions, thereby significantly improving the reconstruction quality of small content regions under a uniform bitrate constraint.

[0161] To further verify the technical effects and feasibility of the present invention, this embodiment performs compression on the input image before compression begins. Simultaneously perform semantic embedding extraction and latent space encoding.

[0162] On the one hand, a pre-trained vision-language model is used to obtain 512-dimensional image embedding vectors. Then load the pre-trained offline PCA parameter file (containing the mean vector). With the principal component matrix C, Mapped to a low-dimensional semantic vector of dimension k:

[0163] ;

[0164] Where k is set to in the actual code. .

[0165] To match the number of latent space channels, a fixed linear mapping matrix is ​​used. Project s into a vector with the same dimension as the latent variable channel, and then expand it into a bias tensor B with the same shape as the latent variable, which serves as the semantic offset for the subsequent Gaussian channel target distribution.

[0166] On the other hand, the input image Input the VAE encoder to obtain latent variables. Its spatial dimensions are The number of channels is C. The compression process does not operate directly on the pixel domain, but rather on the latent space. Perform diffusion noise addition and channel coding.

[0167] In terms of structure awareness, this embodiment utilizes an external semantic annotation file (sem.csv) or a mask file (e.g., an npy file) to construct a Region of Interest (ROI) mask. To achieve structure-aware bit allocation, this invention introduces an ROI mask during the encoding stage. ROI information can come from two sources: one is the external semantic annotation file (sem.csv), which records the mask path and point coordinates for each image; the other is an offline-generated mask image (npy file).

[0168] Specifically, the `_load_sem_csv` function reads `sem.csv` and parses the `image_path`, `mask_path`, and `points` fields line by line. If a `points` field exists (in the format "x,y,w,h;..." or "x,y;..."), the `_points_to_masks_px` function is called to generate soft and hard masks within the downsampled space (e.g., 1 / 8 of the original image). This function first calculates the latent space size based on the image width and height (W,H) and the downsampling factor `down`. Then, a two-dimensional Gaussian weight map is generated for each point or box:

[0169] ;

[0170] A soft mask is obtained by fusing multiple points through normalization and maximization; then, a hard mask is obtained by applying a threshold HARD_THRESH to the soft mask. The soft mask is then subjected to a two-dimensional Gaussian smoothing (the kernel size is determined by SOFT_BLUR_SI) and a power transform SOFT_GAMMA to obtain a smooth probability map in [0,1]. If an external mask file (npy) exists, it is loaded and interpolated to the latent space size using the _maybe_load_mask_from_npy function.

[0171] In the encoding function, the ROI mask roi_mask is converted into a tensor and unified to a dimension. If the size is inconsistent with the latent variable space, nearest neighbor interpolation is used to adjust it. And then expand to The mask is then flattened into a one-dimensional vector and compared with the threshold roi_thresh to obtain the boolean index roi_index_flat.

[0172] ;

[0173] in This indicates a flattening operation. When the ROI ratio is too low (e.g., less than...),... ) or close to 1 (e.g., greater than 1) When roi_index_flat is set to None, it is automatically degraded to single-channel global encoding to avoid the failure of two-channel encoding in extreme cases.

[0174] In the process of structure-aware ROI bit allocation and encoding, at each encoding time step k, the present invention sequentially performs noise prediction, target distribution calculation, semantic bias injection, and ROI two-channel encoding.

[0175] First, based on the current noise latent variable noisy_latent, the time step current_timestep, and the next time step prev_timestep, the noise prediction model is invoked to obtain the noise estimate. It obtains the signal-to-noise ratio between the current time step and the previous time step through its internal SNR function. Then, the prior mean and standard deviation of the current step are calculated using the function P:

[0176] ;

[0177] Then, the function Q is used based on the current noise state latent variable and the target latent variable. Parse the posterior mean:

[0178] ;

[0179] Without semantic prior injection, the standardized target vector is: ;

[0180] When semantic conditions are enabled (i.e., cond_embed is passed and the current time step ratio is given), When the initial injection ratio (cond_ratio=0.955) is greater than the set conditional injection ratio, the low-dimensional semantic vector is first mapped through a linear layer to a bias map (base_bias_map) with the same shape as the latent variables, and then passed through a time-varying weight function. Scaling it up:

[0181] ;

[0182] in, ;

[0183] After normalization by standard deviation, semantic bias is injected into the standardized target vector:

[0184] ;

[0185] And the updated posterior mean is obtained through inverse transformation:

[0186] ;

[0187] To obtain the final standardized target vector Then, it is flattened into a one-dimensional vector q_flat, and the total KL budget for that step is given according to the input manual_dkl_per_step[step_index]. When a valid ROI index roi_index_flat exists and the current step has an explicit KL budget, this invention executes a two-way ROI encoding process:

[0188] Calculate the normalization ratio for the current time step. Through the internal ramp function _ramp( , r) Generate weights within the interval [roi_tstart, roi_tend]. The function defaults to a cosine shape and introduces a front load boost roi_frontload_pow to make the early steps more skewed toward the ROI.

[0189] According to weight Smoothly interpolate the ROI weight coefficients and background weight coefficients from their minimum values ​​to their maximum values:

[0190] ;

[0191] ;

[0192] And crop them to positive numbers (e.g.) Convert the flattened index roi_index_flat to a boolean array idx_roi, and use its inverse set idx_bg as the background index. Calculate the number of elements. , If any value is 0, it degenerates into single-path coding. Otherwise, a weighted sum is constructed:

[0193] ;

[0194] The total KL is proportionally divided into ROIKL and background KL:

[0195] ;

[0196] Based on idx_roi and idx_bg, q_flat is divided into two parts. , Call the encoding interface of the Gaussian channel simulator respectively:

[0197] ;

[0198] ;

[0199] The two sampling results are recombined into a complete sampling vector sample_full by index, then rearranged into a tensor, multiplied by the standard deviation, and added to the prior mean to update the noise latent variable:

[0200] ;

[0201] At the same time, seeds_r and seeds_b, along with their corresponding KL, index, etc., are packaged into chunk_seeds_per_step[step_index] for the current step, for use by the decoding end.

[0202] If an explicit KL budget is not available or the ROI index is invalid at a given step, then q_flat is directly input as the entire Gaussian channel simulator, single-channel coding is performed to obtain the sampling results and seeds, and the noise latent variables are updated in a similar manner. After the entire time series iteration is completed, the encoder outputs chunk_seeds_per_step, dkl_per_step, bytes_per_step, and meta_per_step, which records the ROI / background statistics for each step.

[0203] In the Gaussian channel restoration and diffusion inversion stage at the decoding end, see... Figure 1At the decoding end, the noise latent variable noisy_latent is first initialized according to the image size and time step scheduling. Then, the decoding interface of the Gaussian channel simulator is called using the chunk_seeds_per_step and the total KL log Dkl_per_step output by the encoding end to gradually recover the sampling vector of each time step.

[0204] For steps involving two-way ROI encoding information, `chunk_seeds_per_step[step_index]` contains several components `comps` and an optional background component `bg`; the system pre-constructs a length of [length missing] on the CPU. The zero vector `sample_full` is used to reconstruct the index array for each component based on its stored Boolean index byte stream `idx`. The Gaussian channel simulator is then used to decode the sampling results for the corresponding dimension, which are then filled into the corresponding positions in `sample_full`. The background component is processed similarly. After component filling, `sample_full` is converted into a tensor and reshaped to the same shape as the noise latent variable. Multiplying this tensor by the standard deviation and adding the prior mean yields the noise latent variable for the previous time step.

[0205] For the single-channel coding step, a sample vector of length noisy_latent.numel() is directly decoded using a Gaussian channel simulator, reshaped, and then used for updating. The entire time series is iterated forward until it returns to the initial time step, forming the final noise-state latent variable. .

[0206] After obtaining the latent variables of the final noise state, this invention further utilizes a denoising function guided by spatial semantics to perform diffusion inversion. This function performs pairwise forward inference at each backward time step, obtaining unconditional predicted noise under the same noise state. and conditional prediction noise When enabling the spatial CFG, first calculate the squared difference between the two and then average it along the channel dimension:

[0207] ;

[0208] Subsequently Mini-maximum normalization is performed along the spatial dimension to obtain the high-frequency saliency map. ∈[0,1]:

[0209] ;

[0210] Simultaneously, the previously generated soft semantic mask sem_mask is subjected to bilinear interpolation and smoothing (through two convolutions in the horizontal / vertical directions using a one-dimensional Gaussian kernel) to obtain a smooth semantic map sem∈[0,1], and the effective semantic map is obtained by taking the maximum value of the semantic threshold sem_gate. .

[0211] In the time direction, this invention controls the relative positions of the start and maximum value of the space-based bootstrapping process through the t_start_ratio and t_end_ratio parameters. Let the total number of reverse steps be L, and the current step number be... The normalized time ratio is Then the ramp function is:

[0212] ;

[0213] in, ;

[0214] By combining the base guidance strength (blend_base), soft semantic graph, and high-frequency saliency graph, a spatial guidance graph is constructed:

[0215] ;

[0216] In the latter part of the reverse time, when At that time, further multiply by the subsequent enhancement factor. Then, the entire image is cropped to the [0, alpha_cap] range to obtain the final guide image. Finally, the guided graph is used to perform a pixel-wise linear combination of unconditional and conditional noise predictions:

[0217] ;

[0218] And Substituting the latent variables into the diffusion inversion formula updates the latent variables. After completing the reverse time series traversal, the denoised latent variables are obtained. Then, the VAE decoder outputs the full-image basic reconstruction image. .

[0219] Finally, in the stage of local ROI scaling-up reconstruction and soft fusion, see... Figure 2 To further improve the detail quality of extremely small areas such as text and faces, this invention adds a stage of ROI magnification reconstruction and soft fusion after the full image reconstruction is completed.

[0220] In this stage, one or more rectangular ROI boxes are first determined in the original map space based on the points field in sem.csv or other external annotations. The bounding box is then proportionally expanded using `REFINE_MARGIN` to avoid gaps caused by edge fitting. The `compute_upscale` function is called on each ROI bounding box to calculate the scaling ratio.

[0221] ;

[0222] ;

[0223] in .like Then, the cropped local image is enlarged using bicubic interpolation. .

[0224] Subsequently, the enlarged ROI image is fed into the CLIP model to recalculate the local 512-dimensional semantic embedding, and a new low-dimensional semantic vector and conditional embedding `cond_embed` are obtained according to the aforementioned PCA procedure to ensure that the semantic prior of the local region is more stable. Next, the local image is encoded as latent variables, and the encoding function `encode` is called. A complete Gaussian channel coding is performed under the same time step scheduling and KL allocation `manual_dkl_per_step` as the global one, while recording the noise-state latent variable `noisy_recons` and its time step index corresponding to several candidate reconstruction times `recon_timesteps`.

[0225] During the candidate time evaluation phase, this invention calls the denoise function for each candidate noisy latent variable, using spatial CFG parameters consistent with the global model (such as ZGUIDE_BASE, ZGUIDE_SEM_MAX, ZGUIDE_TSTART, ZGUIDE_TEND, MASK_SMOOTH_K, ALPHA_CAP, etc.) to obtain the corresponding local reconstruction latent variables, and then recovers the local reconstructed image through a VAE decoder. Subsequently, the quality index of each candidate result is calculated according to a preset strategy: when REFINE_TIMESTEP_PICK="best_psnr", the PSNR between the candidate and the original local image is calculated, and the time with the largest PSNR is selected as the optimal time; when REFINE_TIMESTEP_PICK="best_lpips", the LPIPS sensing distance is calculated, and the time with the smallest LPIPS is selected as the optimal time.

[0226] After determining the optimal time step, a complete local denoising process is performed again based on the noise latent variable to obtain the final high-quality ROI reconstructed image. If necessary, the magnified restoration result is restored to the original box size through bicubic interpolation.

[0227] During the fusion phase, this invention generates a two-dimensional soft fusion mask using the `_make_feather_mask` function. For a Region of Interest (ROI) of size (h, w), a two-dimensional mesh coordinate system is constructed, and the Chebyshev distance relative to the center is calculated:

[0228] ;

[0229] in( , Using the center coordinates as the base coordinates, and then constructing a mask based on the edge distance and soft blend radius REFINE_FEATHER_PX=rpx:

[0230] ;

[0231] in The mask's center is close to 1, gradually decaying to 0 at the edges. If a low-resolution soft mask is also provided in sem.csv, it is first interpolated to the ROI size and then multiplied point-by-point with the soft fusion mask to obtain the final fusion weight. ∈[0,1].

[0232] exist Under the guidance of this invention, the image is locally reconstructed. Compared with the initial reconstruction results of the whole map Soft fusion is performed within the ROI region as follows:

[0233] ;

[0234] The merged pixels are then written back to their corresponding positions in the original image, thus visually achieving a local area without obvious edge artifacts.

[0235] To further verify the feasibility and performance of the present invention, the data preparation part of this embodiment uses the kodak and Div2k datasets as the PCA training set and as the image input, which contain 24 and 100 images respectively; the Div2k dataset is cropped to 832×1216 images to ensure consistency with the input of the pre-trained VAE and diffusion model.

[0236] For PCA dimensionality reduction with structural priors, the experimental phase first uses the CLIP model to extract global semantic embeddings (usually 512-dimensional) in batches from a large number of images. Then, sklearn is used to perform a one-time principal component analysis training on this feature set. The number of principal components is usually set to 32 or 64. The mean and principal component matrices obtained from the training are saved as .npz files. When the main process runs, the PCA model is directly loaded for CLIP feature dimensionality reduction without repeated training.

[0237] The ROI region information is centrally managed by the sem.csv file, which supports pixel coordinates, bounding box annotations, and automatic external segmentation (such as .npy format masks generated by the SAM model). The main process automatically parses the annotations and external masks, merges them, aligns them to the original image size, and performs visual inspection. Then, it is scaled to the latent space resolution to participate in encoding and reconstruction, which greatly simplifies the burden of manual annotation and data alignment.

[0238] Diffusion Model: The diffusion model uses mainstream pre-trained models in the industry such as SDXL. The weights are automatically loaded through APIs such as from_pretrained, eliminating the need for local self-training. The accompanying VAE also loads official weights for efficient image and latent variable conversion.

[0239] Configuration file settings: In the main process, the number of steps required for encoding and denoising can be flexibly set in the configuration file (such as SDXL-base.yaml). The commonly used value is 56 steps. ROI weight, margin, target short side, mask parameters, guiding strength, etc. can also be adjusted in the configuration.

[0240] The experimental platform in this embodiment uses the PyTorch framework for model implementation and is tested on an Nvidia RTX 3090 GPU.

[0241] High subjective and objective reconstruction performance: The technical solution of this invention achieves reconstruction quality superior to mainstream diffusion compression methods at various bit rates and resolutions. Figure 3 shows the comprehensive comparison results of various existing technologies on the Kodak dataset, including BD-PSNR, BD-LPIPS, BD-CLIP, and BD-QAlign. This invention has achieved significant advantages in perceptual quality and semantic relevance indicators. In particular, under low bit rate conditions, it can balance global structural consistency and key region detail restoration, and the output image has higher clarity, structural accuracy, and subjective perceptual score. Specific performance comparisons are shown in Table 1.

[0242] Table 1 compares the present invention with existing technologies on the public dataset Kodak;

[0243] Method BD-PSNR↑ BD-LPIPS↓ BD-CLIP↑ BD-QAlign↑ MLIC (ACM 2023) 0.00 0.00 0.00 - / - MS-ILLM (ICML 2023) -1.59 -0.069 +0.013 0.00 CDC (NeurIPS 2023) -10.68 -0.055 +0.012 - / - PerCo (ICLR 2024) -6.27 -0.067 +0.077 +0.188 DiffEIC(TSCVT 2025) -5.24 -0.069 +0.147 +0.166 DiffC (SD1.5) (ICLR 2025) -3.31 -0.069 +0.208 +0.291 PICD (CVPR 2025) -2.03 -0.067 +0.084 - / - This invention -2.09 -0.071 +0.277 +0.316

[0244] Figure 3 shows the rate distortion curves of PSNR, LPIPS, CLIP Score, and Q-Align as a function of bit rate on the Kodak and DIV2K-1024 datasets. The left subplot corresponds to Kodak, and the right subplot corresponds to DIV2K-1024. It can be seen that in the low bit rate region, the curves of this invention are closer to the "ideal direction" as a whole.

[0245] The four subplots in Figure 3(a) are the results applied to the Kodak dataset. From left to right, they are: PSNR – bitrate curve, LPIPS – bitrate curve, CLIPScore – bitrate curve, and Q-Align – bitrate curve, used to compare the performance of several methods in terms of traditional distortion, perceptual quality, and semantic consistency at different bitrates.

[0246] The four subplots in Figure 3(b) are the results applied to the DIV2K-1024 dataset. From left to right, they are: PSNR – bitrate curve, LPIPS – bitrate curve, CLIPScore – bitrate curve, and Q-Align – bitrate curve, which correspond to the comparison of the same set of metrics in high-resolution scenarios.

[0247] Among them, DiffC-SD1.5 / DiffC-SDXL refers to the DiffC diffusion compression baseline (based on SD1.5 and SDXL respectively); PerCo refers to the PerCo perceptual constraint compression method; HiFiC refers to the HiFiC high-fidelity generative image compression method; MS-ILLM refers to the MS-ILLM multi-scale iterative generative compression method; and DiffiC refers to the original DiffC method.

[0248] Region Adaptation and Structure Awareness: Compared with traditional methods, this invention innovatively integrates modules such as dynamic weighting of ROI semantic masks, CLIP-PCA structural prior injection, and spatial adaptive CFG, which significantly improves the protection of details in important regions such as faces and text and the consistency of global content. It performs outstandingly in terms of detail preservation and structural stability, and the restoration of edges, textures, and semantic regions is more natural.

[0249] Subjective reconstruction effects in typical scenarios, for example Figure 4 and Figure 5 As shown (not only to check the restoration effect, but also to check whether the bit rate is low), in magnified areas such as ship rigging and fuselage text, the present invention can still maintain clear outlines and readable text at a bit rate significantly lower than the comparison method, with a significant reduction in artifacts and blur.

[0250] Robustness and Engineering Generality: The method of this invention has been comprehensively tested on various public benchmark datasets and real-world scenarios, demonstrating superior robustness and generalization ability under different compression rates, resolutions, and ROI annotation settings. The algorithm is highly automated throughout the entire process, flexible in configuration, and requires no complex pre-training. It is suitable for various types of data, various terminals, and practical engineering applications, and has good prospects for widespread application.

Claims

1. A diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement, characterized in that, Includes the following steps: Step 1: Semantic prior and latent space representation, i.e., for the input image Extract text features or image semantic features, and use a pre-trained vision-language model to obtain high-dimensional semantic embeddings. And obtain low-dimensional semantic vectors through linear dimensionality reduction. , to transform low-dimensional semantic vectors It is mapped to a conditional bias tensor B consistent with the latent space channel, which is used to adjust the generation distribution during subsequent diffusion. Simultaneously input image Inputting a variational autoencoder yields the initial latent variables that serve as the compression target. ; Step 2: Semantic and region-aware encoding; First, construct a region of interest mask based on semantic information or external annotations. Then, flatten the current noise state latent variable into a one-dimensional vector and divide it into a region of interest index set and a background index set. Then, given the total KL divergence constraint at each step, the KL quotas for the region of interest and the background region are calculated using time-related weighting coefficients, and the total KL is proportionally divided into the region of interest KL and the background KL. The region of interest subvector and the background subvector are encoded separately using a Gaussian channel simulator. After recombination, the noise latent variable is updated according to the forward diffusion formula to obtain the updated noise latent variable. ; Step 3: Denoising and reconstruction guided by spatial semantics; Based on the latent variables of the noise state at the channel state recovery endpoint, a spatial guidance map determined by both semantic mask and high-frequency response is constructed. Unconditional prediction noise and conditional prediction noise are calculated at each inverse time step and combined pixel-by-pixel with weighted summation to obtain a noise estimate with spatial adaptive guidance. The latent variables are iteratively updated along the diffusion inversion trajectory to obtain the denoised latent spatial representation, which is then decoded to recover the full-image basic reconstruction. ; Step 4: Zoom in and enhance details of the region of interest; First, the local ROI magnification and soft fusion module crops one or more regions of interest on the full-image reconstruction based on predefined coordinates or annotation files. After the cropped block is magnified according to the target resolution, the semantic prior construction and encoding and denoising processes are repeated to obtain a high-quality local reconstruction result. For each region of interest, a soft-fusion mask is generated with a center weight of 1 and an edge weight that gradually decays to 0. The local high-quality reconstruction result is then weighted and fused with the initial global reconstruction result within the corresponding region to obtain... ； This improves the reconstruction clarity and structural consistency of small content areas under a uniform bitrate constraint.

2. The diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement according to claim 1, characterized in that, The detailed method for step 1 is as follows: Step 1.1: For the input image as Feature extraction is performed using a pre-trained vision-language model to obtain a high-dimensional semantic embedding vector with a dimension of 512. The loaded principal component analysis parameter matrix and mean vector are subjected to linear transformation PCA to obtain a dimension of low-dimensional semantic vectors Then, using a fixed linear projection matrix, The conditional bias tensor B, which is consistent with the number of latent space channels, is mapped to the target distribution through guiding coefficients during the diffusion coding stage to adjust the subsequent noise sampling distribution. Step 1.2, for the input image as First, the initial latent variables are encoded using a variational autoencoder that matches the diffusion model. , as the target representation of compression; In the forward diffusion process, the initial latent variables According to noise scheduling sequence The noise is gradually increased, and a joint distribution is used to characterize it, as shown in the following expression: ； in, This refers to the cumulative noise parameter; During the encoding phase, when the time step After falling within the preset injection ratio, a set of time-varying guiding coefficients are used. The conditional bias tensor B is additively injected into the normalized target vector, i.e., the calculation is performed. : ； in , The prior mean and standard deviation of the forward diffusion process at the current time step. Based on target latent variables and current noise state The posterior mean obtained through analysis; Then updated to: ; Let represent the standardized target vector obtained at the k-th time step.

3. The diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement according to claim 1, characterized in that, The method for masking the Region of Interest (ROI) in step 2 is as follows: First, the input mask Adjust the dimensions (H, W) to the same height and width as the latent variable space using nearest neighbor interpolation, if a mask is input. For a single channel, it is expanded to (B, 1, H, W) in both the batch and channel dimensions, and then copied along the channel dimension to (B, C, H, W); subsequently, the mask is flattened into a one-dimensional vector and thresholded. Binarization is performed to obtain the Boolean index vector. The remaining positions serve as background indexes. ; Where B represents the batch size, C represents the number of channels, and H and W represent the height and width of the latent variable space, respectively; When Boolean index vector When the proportion of true values ​​is too small or close to 1, it is determined that there is no effective Region of Interest (ROI) mask in the current image, and it degenerates into global single-channel coding.

4. The diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement according to claim 1 or 3, characterized in that, After obtaining the Region of Interest (ROI) mask in step 2, continue with the following steps: At each time step k, the flattened target distribution vector By index Divided into sub-vectors of the region of interest With background subvectors And based on the normalized time ratio With preset start / stop ratio Calculate slope weight Used to generate the ROI weight coefficients for the current step. With background weight coefficient The calculation formula is as follows: ； ； In the above formula, This represents the set of all element locations belonging to the Region of Interest (ROI), used to index and obtain the ROI subvector; This represents the remaining set of background locations, used to index and obtain background subvectors; It is based on the normalized time ratio and start / stop intervals The calculated slope weight; The KL weighting factor assigned to the ROI region at that time step is... and Two hyperparameters passed Obtained through linear interpolation; These are the KL weighting factors assigned to the background region, by , and Linear interpolation is obtained; Let the number of regions of interest and the number of background elements be respectively... , Then construct ； ； and This represents the weighted total of the ROI region and the background region at the k-th time step; Then calculate the total KL budget The formula is: ； in, This is the total KL budget for this time step. and It is based on the former and is proportionally split and allocated to the ROI and background KL constraints to determine how many bits each block can use; The region of interest (ROI) is split into a region of interest (KL) and a background (KL). Then, the ROI and background KL are respectively fed into a Gaussian channel simulator for encoding. , Recombined into a complete sample and based on ； Update the noise latent variables to achieve dynamic bit tilting of the region of interest.

5. The diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement according to claim 1, characterized in that, Step 3 introduces semantic priors and spatial structure information for encoding and denoising. The detailed method is as follows: First, at each backward time step, the normalization ratio r is calculated based on the current step index and the total number of steps, within the interval... Constructing the ramp coefficient using the cosine function , ； This represents the time weighting coefficients obtained based on the cosine curve; Secondly, in the same noise state Predicting noise under the following computational conditions With unconditional prediction noise residual difference of paper Then, average along the channel dimension and normalize to [0,1] in the spatial dimension to obtain the high-frequency saliency map. This is used to reflect the strength of the influence of semantic guidance on different spatial locations in the current step. ； ； It is the basic guiding strength of the spatial guidance diagram Before considering late-stage amplification, it is obtained by weighted fusion of ramp, semantic mask, and high-frequency graph. The weighted map is obtained by normalizing the high-frequency saliency map, which is derived from the difference between conditional noise and unconditional noise, to the 0–1 interval. These are high-frequency weighting coefficients; Next, the smoothed semantic mask With preset gate value according to By fusing semantics and high-frequency joint weights, a basic guidance graph is obtained. : ； This refers to an effective semantic graph; When r > 0.7, for Multiply by the subsequent enhancement factor Finally, cut to Spatial guidance coefficient diagram obtained from the interval ; And perform a pixel-wise linear combination of unconditional and conditional noise predictions, to The original scalar CFG form is replaced in the diffusion inversion update, as shown in the following formula: ; Perform pixel-by-pixel weighted combination, Noise estimates are used as part of the diffusion inversion step to update latent variables.

6. The diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement according to claim 1, characterized in that, The detailed method for step 4 is as follows: For each region of interest bounding box First, calculate the magnification based on its width and height. ； ； ； ； These are the x and y coordinates of the top-left pixel of the region of interest bounding box; These are the x and y coordinates of the bottom right pixel of the region of interest bounding box; Let the magnification factor be... The cropped image is enlarged using bicubic interpolation; Subsequently, the low-dimensional semantic prior of the local image is recalculated, and the encoding function is called under a given encoding time step schedule to obtain the noisy reconstruction latent variable corresponding to each candidate time step. The denoising function is used to invert these candidate latent variables one by one under the guidance of spatial semantics to obtain a set of local reconstructed images. The performance of these images with the original local images in terms of peak signal-to-noise ratio or perceptual similarity index is compared. The time step corresponding to the reconstruction result with the highest PSNR or the lowest LPIPS is selected as the optimal step size. The noisy latent variable under this time step is denoised and inverted again to generate the final local reconstructed image.

7. The diffusion image compression and reconstruction method combining semantic guidance and region detail enhancement according to claim 1 or 6, characterized in that, In step 4, the local region of interest magnification and detail enhancement stage, the final local reconstructed image and the overall basic reconstructed image are fused using a soft fusion mask method. The specific method is as follows: First, a two-dimensional soft-blending mask M is generated based on the local bounding box size (h,w), with a center weight of 1 and gradually decreasing to 0 at the edges. The Chebyshev distance relative to the center is calculated by constructing a row and column grid (x,y). , and according to The rectangular soft fusion weights are calculated. If a low-resolution soft mask is provided externally, the soft mask is first bilinearly interpolated to the local bounding box size, and then multiplied point-by-point with the soft fusion mask before being cropped to [0,1] to obtain the final fusion weight. , local patch image Reconstructed image based on full map The weighted summation within the corresponding regions is performed according to the following formula to obtain the final reconstructed image after fusion. The expression is as follows; ； The merged pixels are then written back to their original positions, thereby improving the reconstruction quality of local detail areas without introducing obvious boundary artifacts.

Images