Image compression reconstruction method based on granularity and diffusion model optimization

Abstract

This invention discloses an image compression and reconstruction method based on granularity and diffusion model optimization, comprising: acquiring an image to be compressed; determining the information density and visual saliency of each region in the image, and generating a multi-level granularity mask accordingly; performing vector quantization encoding on the image to be compressed using the multi-level granularity mask to obtain quantized latent space features; using the multi-level granularity mask as a conditional guide, performing latent diffusion enhancement processing on the quantized latent space features using a denoising network to obtain texture enhancement features, wherein spatial regions corresponding to different granularity levels have heterogeneous denoising scheduling strategies when performing latent diffusion enhancement processing; combining the multi-level granularity mask, spatially aligning and fusing the texture enhancement features and quantized latent space features, and reconstructing based on the fusion result to obtain a reconstructed image. This invention alleviates the homogenization problem of feature reconstruction, achieves on-demand allocation of computing resources at ultra-low bitrates, and is beneficial to improving the visual fidelity and natural perception quality of complex structural regions.

Description

Technical Field

[0001] This invention relates to the field of image data processing and communication technology, and in particular to a high-fidelity image compression and reconstruction method. Background Technology

[0002] Image compression technology is a core support for overcoming the bottlenecks of bandwidth and physical storage resources in massive visual data transmission. With the popularization of high-resolution imaging devices and the surge in demand for complex visual computing, achieving extremely high compression ratios of image features under ultra-low bit rate limited channels, while ensuring that the reconstruction terminal outputs pixel data that conforms to the distribution patterns of human vision and has a clear structure, is of vital technical significance for the underlying data flow in technical scenarios such as space remote sensing, industrial monitoring, and weak network communication.

[0003] Current high compression ratio solutions primarily rely on generative network architectures, especially methods based on discrete vector quantization (DVC). These methods learn a compact, globally discrete feature codebook at the encoder end, mapping continuous images into low-dimensional index sequences for transmission. At the decoder end, a pre-trained decoder is used for lookup table reconstruction. However, in practical engineering applications, the fixed-dimensional spatial sampling grid and globally uniform reconstruction mapping logic often result in insufficient pixel-level fidelity in the reconstructed images. Specifically, the globally uniform discretization process easily erases key high-frequency texture structures, leading to frequent grid stretching and misalignment in the reconstructed image, and easily producing non-realistic generation artifacts that deviate from the original content.

[0004] In summary, existing generative image compression mechanisms, under stringent constraints of ultra-low bitrates, struggle to achieve an effective balance between the structural fidelity of local pixels and the natural perceptual quality of the overall image. The rigid feature truncation of the underlying coding architecture and the homogeneous deductive recovery capabilities of the decoding end make it difficult to converge the distortion problem at complex structural boundaries in the reconstructed image. Therefore, it is necessary to investigate a compressed data processing method that can improve the accuracy of image reconstruction and the adaptability of local features in ultra-low bandwidth scenarios. Summary of the Invention

[0005] Purpose of the invention: To provide an image compression and reconstruction method based on granularity and diffusion model optimization, so as to solve the technical problems of existing generative image compression mechanisms that are difficult to balance local structure fidelity and global natural perception quality under ultra-low bit rate constraints.

[0006] Technical solution: An image compression and reconstruction method based on granularity and diffusion model optimization, comprising: acquiring the image to be compressed; Determine the information density and visual saliency of each region in the image to be compressed, and generate multi-level granularity masks accordingly; The image to be compressed is vector quantized and encoded by combining multi-level granularity masks to obtain quantized latent space features; Guided by the multi-level granularity mask, a pre-trained denoising network is used to perform latent diffusion enhancement processing on the quantized latent space features to obtain texture enhancement features; wherein, the spatial regions corresponding to different granularity levels have heterogeneous denoising scheduling strategies when performing latent diffusion enhancement processing. By combining the multi-level granularity mask, the texture enhancement features and the quantization latent space features are spatially aligned and fused, and the image is reconstructed based on the fusion result.

[0007] Beneficial effects: A dynamic granular scoring mechanism based on information entropy and visual saliency is adopted, combined with cascaded residual codebooks for quantization. Non-uniform bitstream encapsulation is achieved, which tilts the limited transmission bandwidth to high-frequency complex regions, reduces the physical bit overhead of smoothing the background, and ensures the discrete representation accuracy of the core image structure under ultra-low bit rate limited channels.

[0008] A granularity-guided asymmetric diffusion scheduling and progressive freezing denoising mechanism are introduced. Initial noise and computational iteration depth are allocated inversely based on the quantization precision of each region, allowing low-precision regions to achieve sufficient generative texture restoration, while high-precision regions are pre-locked for feature updates, alleviating the limitations of over-smoothing or spurious generation caused by global uniform restoration in traditional models.

[0009] Deformation alignment fusion guided by a two-dimensional sampling space offset field is introduced into the latent space, supplemented by cross-granularity boundary smoothing. Non-rigid creep traction suppresses the inherent mesh stretching and misalignment effects of discrete quantization, and adaptively smooths the computational fault seams generated by heterogeneous diffusion inference. Multi-source prior features are fused and recombined into a unified latent space feature representation, thereby improving the physical continuity of the reconstructed image at the junction of edges and textures. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0011] Figure 2 This diagram illustrates the generation of multi-level granular masks and the cascaded residual codebook waterfall flow retrieval.

[0012] Figure 3 A schematic diagram of heterogeneous denoising scheduling and progressive freezing mechanism in latent diffusion enhancement processing.

[0013] Figure 4 This is a schematic diagram of spatial alignment, fusion, and image reconstruction output. Detailed Implementation

[0014] like Figure 1 As shown in Example 1, the end-to-end complete data closed-loop process of high-fidelity image compression and reconstruction is described.

[0015] Step 101: Obtain the image to be compressed; In digital image processing and communication systems, the image to be compressed is usually the original two-dimensional pixel array acquired by source devices such as digital cameras, mobile terminals or satellite sensors.

[0016] In this embodiment, after acquiring the image to be compressed, the system performs preliminary spatial grid partitioning. Specifically, the input image is divided into multiple non-overlapping image blocks; for example, 16*16 pixels can be used as the basic processing unit. This block processing mechanism aims to transform the global image compression task into local regional feature analysis, thereby providing a spatial positioning basis for subsequent differentiated resource allocation.

[0017] Meanwhile, in order to match the input size requirements of the subsequent latent space network model, if the resolution of the input image is not an integer multiple of the preset downsampling factor (such as f=16), conventional spatial alignment preprocessing such as edge padding or cropping can be performed on it in this step.

[0018] In addition, a pre-trained convolutional neural network can be used to extract forward features for each image patch, mapping the original pixel information into a high-dimensional continuous feature vector, so that subsequent steps can measure the information density of the depth.

[0019] Step 102: Determine the information density and visual saliency of each region in the image to be compressed, and generate a multi-level granularity mask accordingly. In the field of image compression, different spatial regions contribute significantly to visual perception. To establish a quantifiable evaluation system for regional importance, the following approach is used: information density characterizes the complexity of the inherent texture and edges of image patches, while visual saliency characterizes the prior probability that a region will attract human attention.

[0020] After calculating the overall score of each image patch through a specific scoring mechanism, all image patches will be divided into different importance levels according to preset ratio parameters or threshold conditions.

[0021] In this embodiment, the image is divided into three levels, corresponding to the generation of multi-level granularity masks. Physically, this multi-level granularity mask is a two-dimensional discrete array consistent with the spatial arrangement of image blocks. Its matrix elements take values ​​{0, 1, 2}, corresponding to coarse-grained, medium-grained, and fine-grained levels, respectively. Specifically, regions with high redundancy, such as smooth backgrounds and skies, are marked as coarse-grained; regions containing ordinary textures are marked as medium-grained; and core content regions requiring extremely high fidelity, such as faces, complex hair, and sharp edges, are marked as fine-grained. By generating this mask, the system completes the planning blueprint for bit resources on the encoding side.

[0022] Step 103: Combine multi-level granularity masking to perform vector quantization encoding on the image to be compressed to obtain quantization latent space features; Vector quantization coding is the process of mapping continuous image features to discrete codeword indices, and it is a core technique for achieving ultra-low bitrates in generative compression. In this step, vector quantization coding is not applied to the entire image. Figure 1 Instead of treating everyone equally, it is strictly controlled by the aforementioned multi-level granularity mask.

[0023] In practice, the retrieval depth and quantization precision of the feature block in the quantization dictionary are dynamically determined based on the mask value at each position.

[0024] For regions marked with coarse-grained masking, quantization is performed using a shallower search depth or a smaller base codebook to achieve an extremely high compression ratio. For regions marked as fine-grained, a deeper cascaded search is activated or a high-precision, large-capacity codebook is invoked to approximate the original features as closely as possible. After this differentiated quantization process, continuous features are transformed into a series of discrete codebook indices.

[0025] To facilitate network transmission or storage, the index sequence, together with a multi-level granularity mask, is encapsulated into a bit data stream using lossless entropy coding techniques such as adaptive arithmetic coding.

[0026] At the decoding end, after receiving and parsing the data stream, these discrete indices are remapped back to the continuous latent space through a lookup table operation, thereby recovering the basic continuous latent space feature tensor, i.e., the quantized latent space feature.

[0027] At this point, the quantization latent space features contain the basic structure of the image, but information loss or grid artifacts are inevitable in low-precision quantization regions.

[0028] Step 104: Guided by the multi-level granularity mask, perform latent diffusion enhancement processing on the quantized latent space features using a pre-trained denoising network to obtain texture enhancement features; wherein, the spatial regions corresponding to different granularity levels have heterogeneous denoising scheduling strategies when performing latent diffusion enhancement processing. After obtaining the quantized latent space features that include quantization errors, directly decoding them often results in false details or distortion in the generated image. Therefore, this embodiment introduces a diffusion model to perform feature repair and detail compensation in the latent space dimension.

[0029] When performing latent diffusion enhancement processing, the pre-trained denoising network not only receives the noisy latent space features, but also injects multi-level granularity masks as conditional guidance information into the network.

[0030] The most crucial mechanism lies in the implementation of a heterogeneous denoising scheduling strategy. That is, the denoising network no longer performs a globally uniform Markov inference on the entire feature map, but instead customizes different computation paths for different regions based on the spatial distribution of multi-level granularity masks.

[0031] For coarse-grained regions with low quantization accuracy and severe information loss, the denoising network will allocate stronger generative computing power to them, prompting them to generate visually reasonable texture content to fill the gaps. For fine-grained regions with high quantization accuracy and well-preserved structure, the denoising network limits the generation amplitude to maintain the original real details, thereby avoiding the introduction of illusory features that do not belong to the original image.

[0032] After spatiotemporal heterogeneous scheduling that allocates repair intensity on demand, the output tensor not only repairs quantization defects but also maintains fidelity, thus obtaining the texture enhancement features.

[0033] Step 105: Combine the multi-level granularity mask to perform spatial alignment fusion of the texture enhancement features and the quantization latent space features, and reconstruct the image based on the fusion result.

[0034] Since quantization latent space features originate from discrete gridded lookup tables, while texture enhancement features originate from continuous diffusion generation, there is often a slight misalignment between the two at the pixel level. Simply superimposing them linearly can easily produce halos or jagged edges in the reconstructed image. Therefore, spatial alignment fusion must be performed before inputting them into the final decoder.

[0035] Specifically, the system again utilizes the boundary and transition information provided by the multi-level granularity mask to calculate the spatial offset guidance for feature fusion. During the fusion process, the quantized latent space features are guided to actively and non-rigidly align with the sharp structures in the texture enhancement features. After structural alignment and adaptive weight synthesis, the dual-domain features are integrated into a latent space feature represented by a fused feature representation.

[0036] The fusion result is fed into a decoder built on a generative adversarial network (GAN) or a convolutional neural network, which maps the latent space back to the pixel space, outputting a high-fidelity, visually natural reconstructed image.

[0037] The system architecture of this embodiment breaks through the limitation of isolated operation of each module in traditional solutions by globally connecting granular masks at the three key nodes of encoding, diffusion and fusion, and realizes dynamic resource control in bandwidth-constrained environments.

[0038] like Figure 2As shown in Example 2, this example describes the core technical steps of how to resolve dimensional conflicts through normalization and nonlinear weighting, thereby accurately evaluating the information density of image blocks.

[0039] Step 201: Spatial block processing is performed on the image to be compressed to obtain multiple image blocks. Unstructured information entropy and visual saliency score are extracted from each image block in the image to be compressed. After acquiring the image to be compressed and dividing it into multiple image blocks, it is necessary to quantize the feature complexity of each local area.

[0040] Unstructured information entropy is used to measure the degree of disorder in the distribution of pixel grayscale values ​​or feature vectors within an image patch. The process is as follows: the probability of occurrence of each feature value within the image patch is statistically analyzed, and the Shannon information entropy is calculated using the probability distribution formula. A larger value indicates richer high-frequency texture or edge details in the region.

[0041] Furthermore, visual saliency scores are used to characterize the prior probability that a specific region will attract the attention of the human visual system. The system inputs image patches into a pre-trained saliency extraction network for forward propagation and outputs the saliency intensity value corresponding to that region.

[0042] The pre-trained saliency extraction network is a visual saliency detection model known in the art. Those skilled in the art can use publicly available saliency detection methods, such as multi-scale attention mechanisms based on feature pyramids, and publicly available datasets for training. The specific network parameter configuration can be adaptively adjusted according to actual accuracy requirements.

[0043] For example, the pre-trained saliency extraction network can adopt a feature pyramid architecture and output a continuous probability value through multi-scale feature fusion.

[0044] Step 202: Map the unstructured information entropy and the visual saliency score to preset dimensionless intervals to obtain normalized information entropy and normalized saliency score; Since the physical dimension of unstructured information entropy is usually bits, its numerical range fluctuates greatly, while visual saliency scores are usually designed to fall within a specific probability interval. If the two are directly linearly superimposed, the one with the larger numerical range will dominate the result, rendering the resource compression allocation strategy ineffective.

[0045] Therefore, a dimension unification operation is required. The system uses a mapping function to project the extracted unstructured information entropy and visual saliency score into preset dimensionless intervals. For example, the system can use a maximum-minimum normalization method to uniformly map the values ​​of the two dimensions to a closed interval between 0 and 1.

[0046] After this mapping operation, the two no longer have their original physical units and become purely relative intensity representation parameters, which yields the normalized information entropy and normalized significance score, thus providing a unified numerical benchmark for subsequent coupling calculations.

[0047] Step 203: Use a preset exponential gating function to perform a nonlinear weighted calculation on the normalized information entropy and the normalized saliency score to obtain the comprehensive score of each image block; After obtaining the dimensionless basic parameters, in order to strengthen the coupling guidance of information entropy and visual saliency in compressed resource allocation, the system adopts an exponential gating mechanism instead of simple linear superposition.

[0048] When a region possesses high-frequency texture and is located at the core of human eye gaze, its overall score experiences a non-linear surge. In this calculation process, the system introduces a learnable temperature parameter, which is used to scale the normalized information entropy, resulting in a temperature-modulated normalized information entropy, thereby dynamically adjusting the sensitivity of the non-linear weighting.

[0049] Meanwhile, to prevent numerical overflow caused by exponential operations in extremely high-entropy regions, the system performs a truncation operation on the output value of the exponential gating function, limiting its maximum amplification factor to a preset constant range, and obtaining the truncated gating weighted value. The truncated gating weighted value is used to calculate the comprehensive score.

[0050] The overall score S_i of the i-th image patch: S_i=λ*H_norm_i+(1 -λ)*V_norm_i*min(exp(H_norm_i / τ)e 2 ); Where λ is a preset linear weighting coefficient, H_norm_i is the normalized information entropy of the i-th image patch, V_norm_i is the normalized significance score of the i-th image patch, exp() is the natural exponential function, τ is a learnable temperature parameter used to adjust sensitivity, min() is the minimum function, and e 2 A preset truncation upper limit constant to prevent numerical overflow.

[0051] In some alternative implementations, if the computing power of the current computing device is subject to extremely strict constraints and it is permissible to sacrifice some nonlinear accuracy in resource allocation, the system may also employ a basic linear weighted alternative.

[0052] Specifically, the system directly uses preset weights to perform multiplication and addition operations on the normalized information entropy and the normalized significance score. That is, it calculates the product of the preset first weight and the normalized information entropy, calculates the product of the preset second weight and the normalized significance score, and then sums the two products and outputs them as the comprehensive score.

[0053] Step 204: Based on the comprehensive score and the preset granularity ratio parameter, determine the granularity level corresponding to each image block and generate a multi-level granularity mask.

[0054] After obtaining the comprehensive score of all image patches in the entire image, it needs to be converted into discrete levels to guide the coding and diffusion sides. The system pre-configures a set of granularity scaling parameters based on the set bit rate target.

[0055] For example, users can configure the percentage of area occupied by fine-grained, medium-grained, and coarse-grained pixels in the entire image to be 20%, 30%, and 50%, respectively. The overall scores of all image patches are sorted in descending order, with the top 20% of image patches assigned the fine-grained level; image patches with scores between 20% and 50% are assigned the medium-grained level; and the remaining image patches are assigned the coarse-grained level.

[0056] After allocation, the system reassembles the level labels into a two-dimensional array according to the original spatial coordinates of the image patches, outputting a complete multi-level granularity mask. This multi-level granularity mask forms a control matrix, which is used to directly instruct the search depth of the subsequent quantization dictionary and the number of denoising iterations for the diffusion model.

[0057] like Figure 3 As shown in Example 3, the technical implementation process of the waterfall-style retrieval path and non-uniform code stream encapsulation of the multi-level dictionary is described.

[0058] Step 301: Provide a pre-built concatenated residual codebook, which contains multiple quantization dictionaries nested in a hierarchical manner; In the feature discretization stage of image compression, the system relies on a pre-constructed quantization reference template. In this embodiment, a cascaded residual codebook structure is used instead of a single codebook. The cascaded residual codebook is composed of multiple quantization dictionaries located in different feature spaces, connected in series, to meet the differentiated quantization accuracy requirements of different granularity regions.

[0059] During implementation, a quantization dictionary with three levels is pre-built.

[0060] The capacity of the first-level coarse-grained codebook is configured to 256, which is used to extract large-scale structural features of the image basis.

[0061] The capacity of the second-level refinement dictionary is configured to 512, used to compensate for edge details at a medium scale.

[0062] The capacity of the third-level large-capacity dictionary is configured to 1024, which is used to capture high-frequency, minute texture features.

[0063] Each level of the quantization dictionary contains a specific number of feature vectors, and each feature vector has the same channel dimension, thereby enabling multi-scale nested discretization representation of continuous features.

[0064] The latent space features of the image to be compressed are obtained by using a pre-trained encoder to encode the features of the image to be compressed.

[0065] Step 302: Perform waterfall-style retrieval on the latent space features of the image to be compressed, calculate the reconstruction residual after quantization at the current level, and determine whether the reconstruction residual should enter the next level of quantization dictionary for iterative retrieval based on the granularity level indicated by the multi-level granularity mask. Traditional residual vector quantization typically performs the same-depth retrieval computation across all spatial locations of the feature map, resulting in computational redundancy. This scheme introduces a waterfall-style retrieval mechanism, utilizing multi-level granularity masks as conditional gating signals. After latent spatial features pass through the current level of the quantization dictionary, the system calculates the difference between the input feature and the found nearest-neighbor codeword to obtain the reconstructed residual. The granularity level indicated by the multi-level granularity mask is read pixel-by-pixel as the criterion for whether to pass the reconstructed residual to the next level of the quantization dictionary. This mechanism ensures that limited codebook retrieval computational power is allocated only to complex regions with high information density.

[0066] Step 303: The latent space features of the image to be compressed are retrieved in the first layer using the first-level coarse-grained codebook, and the first-level reconstruction residual is calculated. At the starting point of the waterfall-style retrieval, the latent space features contain information from all locations across the entire map. The system calculates the Euclidean distance between continuous latent space features and each codeword in the first-level coarse-grained codebook, selects the codeword with the smallest distance for reconstruction, and calculates the difference before and after reconstruction. The above residual extraction process is implemented using the following formula: E_1 = Z - Lookup(CB_coarse, Z); Here, E_1 is the first-level reconstruction mapping residual map, Z is the continuous latent space feature, Lookup() is the nearest neighbor matching retrieval function, and CB_coarse is the first-level coarse-grained codebook. Through this step, all spatial locations obtain the most basic discretized representation, while the lost high-frequency information is preserved in the first-level reconstruction mapping residual map.

[0067] Step 304: Based on the multi-level granularity mask, input the first-level reconstruction residuals corresponding to the medium and fine-grained regions into the second-level refinement dictionary for two-level retrieval, and calculate the second-level reconstruction residuals; After obtaining the first-level reconstructed mapping residual map, the system performs spatial dimension routing based on multi-level granularity masks. The system detects coordinates in the mask matrix with values ​​greater than or equal to 1, i.e., medium-granularity and fine-granularity regions. For these specific locations, the system extracts the corresponding feature vectors in the first-level reconstructed mapping residual map and inputs them into the second-level refined dictionary for lookup matching. After matching, the system calculates the difference between the current input residual and the selected codeword in the second-level dictionary to obtain the second-level reconstructed mapping residual map. Coarse-grained regions with mask values ​​of 0 are skipped in this step and no longer participate in the second-level retrieval; their feature states are locked.

[0068] Step 305: Input the second-level reshoot residual corresponding to the fine-grained region into the third-level large-capacity dictionary for three-level retrieval.

[0069] Further detection is performed on the coordinates of the multi-level granularity mask where the value equals 2, identifying the fine-grained regions. For these core regions requiring extremely high fidelity, the corresponding quadratic micro-residues in the second-level reconstruction mapping residual map are extracted and fed into the third-level large-capacity dictionary for nearest neighbor matching. At this point, the fine-grained regions have completed the deepest level of quantization compensation, achieving the highest precision discrete feature representation.

[0070] Step 306: Combine the codebook indexes obtained from each level of retrieval to obtain the quantized latent space features.

[0071] After the aforementioned multi-level truncation retrieval, a set of codebook index sequences of varying lengths is generated for each spatial location. The first-level index generated in the coarse-grained region, the first and second-level indices generated in the medium-grained region, and the first, second, and third-level indices generated in the fine-grained region are concatenated according to spatial coordinates to form a multi-channel combined discrete index. This combined index is reconstructed by inversely summing the corresponding codebooks at each level, thereby generating the aforementioned quantization latent space features at the decoding end.

[0072] Step 307: Determine the codebook lookup depth corresponding to each position in the cascaded residual codebook based on the granularity level corresponding to each position in the multi-level granularity mask. To achieve the final bitstream encapsulation, the system establishes a deterministic mapping relationship between granularity level and codebook lookup depth.

[0073] Specifically, the coarse-grained region with a granularity level configured as 0 has a codebook lookup depth of 1 layer. The medium-granularity region, with a granularity level configured as 1, has a codebook lookup depth of 2 layers. The granularity level is configured as a fine-grained region of 2, and its codebook lookup depth is determined to be 3 layers. This depth parameter defines the physical boundaries of subsequent bitstream encapsulation truncation operations.

[0074] Step 308: According to the index bit width corresponding to the codebook lookup depth, encapsulate the codebook index at each position with a non-uniform code stream; to obtain a non-uniform encoded code stream, which is used for transmission or storage. In traditional image compression protocols, bitstreams are typically encapsulated as uniform matrices.

[0075] In this embodiment, the system employs a layered dynamic encapsulation strategy. The system calculates the basic index bit width based on the capacity of the first-level coarse-grained codebook. The first-level dictionary with a preset capacity of 256 corresponds to an 8-bit index bit width, the second-level dictionary with a capacity of 512 corresponds to a 9-bit index bit width, and the third-level dictionary with a capacity of 1024 corresponds to a 10-bit index bit width. When constructing the bitstream, the system first writes an 8-bit first-level index stream for all positions in the entire graph.

[0076] Only for positions with a search depth greater than or equal to 2, an additional 9 bits of the second-level index stream are appended.

[0077] The system appends 10 bits of a third-level index stream only to the location with a search depth of 3. The final encapsulated bitstream is structurally composed of header information, arithmetic-coded multi-level granularity mask streams, and the aforementioned non-uniformly distributed index streams concatenated.

[0078] Step 309, wherein the higher the granularity level, the deeper the codebook lookup depth, and the number of bits occupied by different spatial positions in the non-uniform coding bitstream changes dynamically with the granularity level.

[0079] Assuming the initial resolution of the input image is 256 by 256 pixels and the encoder downsampling factor is configured to 16, the resolution of the latent space features is 16 by 16, totaling 256 spatial locations. Let the area proportions of coarse-grained, medium-grained, and fine-grained regions after granularity evaluation be 50%, 30%, and 20%, respectively.

[0080] The first level contains indexes for all positions, occupying 256 * 8 = 2048 bits of data.

[0081] The second layer contains indexes for medium-granularity and fine-granularity locations, accounting for 50% of the total area and occupying 256 * 0.5 * 9 = 1152 bits of data.

[0082] The third layer contains only fine-grained location indexes, accounting for 20% of the total area and occupying 256 * 0.2 * 10 = 512 bits of data.

[0083] After compression by an adaptive arithmetic encoder, the multi-level granularity mask array theoretically occupies approximately 406 bits of data. Summing the above terms, the total data volume of the non-uniform bitstream is 2048 + 1152 + 512 + 406 = 4118 bits, calculated based on the information entropy estimation of the ternary mask under a given probability distribution and the actual efficiency of arithmetic coding. This data volume corresponds to a compression ratio of approximately 0.063 bits per pixel.

[0084] As an alternative, if the system adopts the traditional uniform depth residual quantization strategy, requiring all 256 positions to perform a 3-layer complete search, the total data volume would be 256*27+406=7318 bits, since the sum of the index bit widths at each position is 8+9+10=27 bits. Comparative calculations show that the non-uniform bitstream encapsulation scheme provided in this case reduces the total bit rate by approximately 43.7% while maintaining the required index accuracy in high-frequency detail regions. This objectively achieves a technical improvement in compression efficiency under ultra-low bit rate transmission bandwidth constraints.

[0085] like Figure 4 As shown in Example 4, the computational steps for resolving the spatial distribution trade-off between fidelity and generation capability in the diffusion model are described using a heterogeneous denoising scheduling strategy.

[0086] Step 401: Determine the initial noise intensity and target denoising depth corresponding to each spatial location based on the granularity level indicated by the multi-level granularity mask; Before proceeding with latent diffusion enhancement, the system needs to configure spatial scheduling parameters on a temporal scale for the pre-trained denoising network. During this process, a multi-level granularity mask serves as a control signal source, guiding the time step allocation for each spatial coordinate point across the entire image.

[0087] Internally, a set of time steps is pre-defined, which is divided downwards from the maximum total number of iterations. This set contains three levels of target denoising depth values: high, medium, and low.

[0088] The spatial matrix of multi-level granularity masks is traversed to extract the granularity level label recorded at the current coordinate point. A numerical mapping function is then used to bind the corresponding target denoising depth to this coordinate point, and the initial noise intensity accumulation coefficient is associated with this depth value. Unlike the conventional mode of denoising networks that perform homogenized processing on all image features, this one-to-one binding operation of spatial location and temporal step depth establishes the fundamental conditions for asymmetric feature processing. The specific value of the initial noise intensity accumulation coefficient is determined by a preset noise scheduling strategy. Those skilled in the art can use conventional diffusion model noise scheduling methods such as linear scheduling and cosine scheduling to obtain the corresponding accumulation coefficient value by looking up a table or calculating based on the target denoising depth.

[0089] Step 402, wherein the granularity level is negatively correlated with the initial noise intensity and the target denoising depth, so that image regions with lower quantization accuracy obtain higher initial noise and a deeper number of denoising iterations.

[0090] In this step, a negative correlation mapping strategy is enforced. Specifically, the coarse-grained level with a value of 0 in the multi-level granularity mask corresponds to the region with the most high-frequency information stripped in the pre-quantization stage, and is assigned the highest target denoising depth and the highest initial noise intensity. The fine-grained level with a value of 2 in the multi-level granularity mask preserves the dense original image texture, and is assigned a very small target denoising depth and the minimum initial noise intensity.

[0091] Through the reverse mapping mechanism, regions with low quantization accuracy are heavily denoised in subsequent processing, and most of the residual features are covered by random Gaussian noise, thus prompting the denoising network to reconstruct visually reasonable edge textures in a long reverse inference process; while fine-grained regions with high quantization accuracy are only subject to slight feature perturbations, and discrete artifact elimination is completed in a very short iteration cycle, preserving the structural features of the original signal.

[0092] Step 403: Using the quantized latent space features as anchor points, inject random noise sampled from the standard normal distribution into the latent space according to the initial noise intensity to obtain the spatially heterogeneous initial noise latent features; After configuring the spatial scheduling parameters, asymmetric noise initialization is performed at the beginning of the reverse denoising inference. Instead of using pure Gaussian noise as the starting point for inference, the system extracts quantized latent space features as the baseline structure anchor point and combines them with a random noise matrix sampled from the standard normal distribution for linear interpolation mixing.

[0093] During the mixing process, the system uses the initial noise intensity accumulation coefficient determined in the previous steps to perform a spatially weighted summation of the reference structure anchor points and the random noise matrix, pixel by pixel. The formula for this calculation process is expressed as: Z_init(x, y)=α_T(x,y)*Z_base(x,y)+(1-α_T(x,y))*ε(x,y); Where Z_init(x, y) is the initial latent noise feature at spatial coordinates (x, y), Z_base(x, y) is the quantized latent spatial feature at spatial coordinates (x, y), ε(x, y) is the random noise sampled from the standard normal distribution at spatial coordinates (x, y), and α_T(x, y) is the initial noise intensity accumulation coefficient determined by the target denoising depth corresponding to spatial coordinates (x, y). After this calculation, the output feature tensor exhibits a heterogeneous state in the spatial domain. The coordinate region corresponding to the coarse-grained feature exhibits a highly disordered noise distribution, while the coordinate region corresponding to the fine-grained feature retains the original local contour.

[0094] In one optional implementation, a cosine noise scheduling strategy is adopted, with α_T value of 0.95 for fine-grained regions, α_T value of 0.7 for medium-grained regions, and α_T value of 0.2 for coarse-grained regions.

[0095] Step 404: Using the pre-trained denoising network, perform multi-step reverse denoising iterations on the initial noise latent features to obtain the texture enhancement features.

[0096] The system inputs spatially heterogeneous initial noise latent features into a pre-trained denoising network and initiates Markov inverse denoising computation in descending order of time steps. The pre-trained denoising network employs a conditional convolutional architecture with skip connections, and its encoding path includes four progressively downsampling stages, with the channel dimension increasing sequentially to 512. In a single iteration of the multi-step inverse denoising iteration, the network predicts the residual noise components contained in the current input features and approximates continuous features through weighted subtraction operations. Furthermore, multi-level granular masks and quantized latent spatial features are concatenated according to the channel dimension to form a complete conditional vector, which is continuously injected into each intermediate layer of the denoising network through cross-attention layers, guiding the convergence of the denoising direction.

[0097] Step 405: In each round of reverse denoising iteration, determine whether the current iteration step has reached the target denoising depth corresponding to each spatial location; Within the derivation loop of the multi-step reverse denoising iteration, an independent effective iteration counter is maintained for each spatial location to record the number of denoising updates actually performed at that location. After each single-step iteration calculation, the effective iteration counter of the currently active location is incremented, and it is determined whether the effective iteration count value of each spatial location has reached its pre-bound target denoising depth. The current global time step value is extracted and compared with the pre-bound target denoising depth of each spatial location.

[0098] If the current global time step value is greater than the target denoising depth corresponding to a certain coordinate position, it is determined that the position has not yet entered its set active update window; if the current global time step value is less than or equal to the target denoising depth corresponding to the coordinate position, it is determined that the position has entered the active update window, and the current output data of the denoising network needs to be absorbed and retained.

[0099] Step 406: For the spatial location that has reached the target denoising depth, generate a spatial freeze mask based on the judgment result, and use the spatial freeze mask to lock the current feature value at that location to obtain the stable state feature. Based on the numerical comparison results of the preceding steps, the system constructs a dynamically updated spatial frozen mask tensor in memory, which is used to perform divide-and-conquer update computation at the feature map level.

[0100] When the effective iteration count value at a certain spatial coordinate position reaches its target denoising depth, the effective denoising cycle at that position is completed and the feature locking mechanism is triggered.

[0101] The output value calculated by the denoising network at this time step is extracted as the anchoring reference, and in all subsequent decreasing rounds, the data update path at this coordinate position is blocked using a spatial freeze mask.

[0102] Specifically, the mask coefficients corresponding to the locked positions are set to 0, and the mask coefficients corresponding to the active positions are set to 1. Then, the predicted features currently output by the denoising network are weighted and calculated with the extracted anchoring reference features. The formula for this feature locking calculation process is expressed as: Z_current(x,y)=M_active(x,y)*D_out(x,y)+(1-M_active(x, y))*Z_anchor(x,y); Where Z_current(x, y) is the current time step feature after merging and updating, M_active(x, y) is the spatial active mask parameter of the current time step, D_out(x, y) is the predicted feature output by the denoising network at the current time step, and Z_anchor(x, y) is the stable state feature extracted and saved when the feature locking mechanism is triggered at the corresponding spatial coordinate position.

[0103] Thus, the fine-grained region maintains a locked, stable feature state during multiple rounds of denoising derivation, while the coarse-grained region continuously receives newly synthesized texture pixels from the denoising network.

[0104] Step 407: Use the stable state features as spatial context to guide spatial locations that have not reached the target denoising depth to continue feature updates.

[0105] The spatial coordinate pixels locked as stable-state features are not removed from the computation network. In subsequent denoising iterations, the complete feature tensor formed by merging active and stable features is input into the denoising network again. At this point, the pre-frozen fine-grained features act as constraints within the receptive field of the local convolutional kernels of the denoising network, serving as static spatial context structure reference points and participating in feature aggregation and gradient backpropagation calculations at adjacent active locations.

[0106] This spatial constraint mechanism controls the pixel expansion boundaries of coarse-grained regions when generating texture content, maintaining the semantic coherence and boundary smoothness of the global feature map in the spatial dimension.

[0107] In some alternative implementations, for terminal devices with limited computing memory and hardware environments that do not support high-dimensional latent space matrix derivation, the quantized latent space features can be directly inversely mapped to the pixel value space through a conventional basic decoding module to generate an initial pixel image matrix containing quantization errors.

[0108] Building upon this, the aforementioned asymmetric scheduling and progressive freezing reverse denoising process is directly executed within the two-dimensional pixel space. This alternative increases the frequency of memory read / write operations during the spatial resolution restoration stage, but it can directly load conventional pixel-level pre-trained model weights, reducing the computational cost of offline construction of dedicated latent domain denoising models.

[0109] Example 5 describes how to eliminate feature seam artifacts caused by differences in spatial heterogeneous diffusion depth through local feature harmonic convolution.

[0110] Step 501: Calculate the gradient magnitude of the multi-level granularity mask, and determine the cross-granularity boundary band based on the gradient magnitude; Because the progressive freeze mechanism implements heterogeneous denoising depth scheduling in space, adjacent local regions of the image may undergo denoising steps with huge differences, resulting in abrupt changes in the statistical distribution of features and forming granular seam artifacts.

[0111] To accurately locate these discontinuous regions, the system needs to calculate the spatial gradient of multi-level granularity masks. Considering that direct differentiation of discrete multi-level granularity masks will produce impulse distributions, the system applies a Gaussian smoothing filter to them.

[0112] For example, a Gaussian kernel with a standard deviation of 1.5 pixels is used for convolution preprocessing to transform the discrete step signal into a smooth transition state. After smoothing preprocessing, the system uses an edge detection operator to calculate the partial derivatives of the feature map in the horizontal and vertical directions.

[0113] Specifically, the system uses preset edge detection operators to extract spatial gradient matrices in orthogonal directions and calculates the vector magnitudes of both to obtain the gradient magnitude at each spatial coordinate point. After obtaining the global gradient magnitude distribution, the system sets a threshold to filter out transition regions.

[0114] For example, the threshold can be set to 30% of the global maximum gradient magnitude. The system marks spatial coordinate points with gradient magnitudes greater than this threshold as active, and these consecutive sets of activated coordinate points together constitute a cross-granularity boundary band with a width of approximately 3 to 5 latent space pixels. Coordinate points in non-boundary regions are marked as silent and do not participate in subsequent feature reconciliation calculations.

[0115] Step 502: In the texture enhancement feature, the features located within the cross-granularity boundary band are smoothed using local feature harmonic convolution to obtain a smoothed texture enhancement feature, which is used for the spatial alignment fusion. After locking the spatial coordinate range across the granularity boundary band, the system triggers smoothing calculations of the feature dimensions within this region. Specifically, after each iteration of the reverse denoising iteration and feature freezing operation, the system extracts the current output feature tensor. A convolutional kernel with a specific receptive field is configured, such as a Gaussian smoothing kernel with a size of 3*3 and a standard deviation of 1.0. The system performs the sliding calculation of this convolutional kernel only for feature pixels located within the granularity boundary band and linearly weights and mixes the convolutional output with the original features.

[0116] The formula for the above local feature harmonic convolution process is: Z_blend(x,y)=(1-η(x,y))*Z_prev(x,y) η(x,y)*Conv_K(Z_prev(x,y)); Where Z_blend(x, y) represents the blended feature at spatial coordinates (x, y) after smoothing, η(x, y) represents the blending intensity parameter at spatial coordinates (x, y), Z_prev(x, y) represents the original input feature at spatial coordinates (x, y) before smoothing, and Conv_K() represents the convolution extraction operation performed on local features using a preset Gaussian kernel. For silent coordinate points not covered by cross-granularity boundary bands, their feature values ​​remain unchanged, i.e., the above convolution and weighting calculations are skipped, and the original input features are directly passed downstream.

[0117] Step 503, wherein the mixing intensity of the smoothing process is adaptively determined by the gradient magnitude, and is used to eliminate feature discontinuities generated by different denoising scheduling strategies.

[0118] In linear weighted mixture computation, the mixture intensity parameter controls the weight ratio of the original features to the convolutional smoothed features. To prevent excessive blurring caused by fixed weights in low gradient regions, the system establishes a positive correlation between the mixture intensity parameter and the gradient magnitude extracted in the previous step.

[0119] Specifically, the gradient magnitude at the current coordinate point is divided by the global maximum gradient magnitude to obtain the normalization ratio. This normalization ratio is then multiplied by a preset maximum blending intensity limit to output the adaptive blending intensity parameter for the current coordinate point. To ensure that the original input features always dominate the blending result and avoid the loss of underlying structural information, the maximum blending intensity limit is typically set to 0.5. Through this adaptive adjustment mechanism, regions with steeper seams will receive a stronger smoothing effect, thereby eliminating texture breaks and brightness jumps caused by excessive differences in the number of diffusion steps between adjacent regions in the latent space dimension.

[0120] Example 6 describes the core technical steps of using the offset field operator to achieve deformation alignment and fusion of discrete structural features and diffusion texture features.

[0121] Step 601: Generate a gradient boundary matrix based on the local level transition of the multi-level granularity mask; Before spatial assembly of features, it is necessary to accurately locate the physical coordinate intervals where structural misalignment may exist between discrete features and continuously generated features. Multi-level granularity masks, because they record the spatial distribution of different coding precisions, have boundaries where their values ​​undergo abrupt changes, which are regions where the quantization grid stretching effect is significant. A two-dimensional convolution operation is performed on the spatial matrix of the multi-level granularity mask using an edge detection filter. Specifically, the edge detection filter can employ the Sobel operator or the Prewitt operator. The system uses the edge detection filter to calculate the partial derivatives in the horizontal and vertical directions to extract the location information of local level jumps. The system marks coordinate points where the magnitude of the partial derivative exceeds a preset threshold as valid boundaries, thereby generating a binary distribution matrix or a continuous distribution matrix consistent with the spatial resolution of the original input features, i.e., the gradient boundary matrix. This matrix serves as a spatial attention control parameter in subsequent calculations, limiting the deformation alignment calculation process to be activated only at specific overlapping and blurred edges.

[0122] Step 602: Based on the gradient boundary matrix, at the overlapping edge of the texture enhancement feature and the quantization latent space feature, the offset prediction network is used to generate the learned two-dimensional sampling space offset field. After locating the overlapping edges that need correction, the continuous high-frequency detail tensor contained in the texture enhancement features and the discrete reconstructed structure tensor in the quantized latent space features are extracted. These two feature tensors are concatenated along the channel dimension, and the concatenated tensor is then element-wise multiplied with the gradient boundary matrix. This operation filters out redundant information from the smoothing background region.

[0123] An offset prediction network comprising multiple cascaded convolutional layers is used to perform forward propagation calculations on the filtered fused tensor, outputting a feature map with a channel dimension of 2. The first channel of this feature map corresponds to the absolute value of the horizontal coordinate offset in space, and the second channel corresponds to the absolute value of the vertical coordinate offset in space. These absolute coordinate offset values ​​together constitute a set of learned two-dimensional sampling spatial offset fields, which are used to record the two-dimensional vector direction and displacement distance that the underlying discrete representation should undergo in the spatial grid.

[0124] Step 603: Using the two-dimensional sampling space offset field, guide the quantized latent space feature to be aligned with the edge structure of the texture enhancement feature to obtain the aligned quantized latent space feature.

[0125] Step 604: When performing feature fusion, a deformation attention kernel is applied so that when the quantized latent space features are merged, the sampling points are driven by the two-dimensional sampling space offset field to non-rigidly move towards the sharp edge pixels corresponding to the texture enhancement features, thereby obtaining the aligned quantized latent space features.

[0126] In standard convolutional feature extraction operations, the sampling grid of the receptive field is fixed and rigid, and cannot adapt to spatial misalignment at the feature level.

[0127] In this embodiment, the system introduces an elastic correction calculation mechanism, replacing the standard regular sampling grid with a dynamic sampling grid with coordinate offsets. Specifically, the system uses quantized latent space features as the basic operation object, superimposing the coordinate offset vectors recorded in the two-dimensional sampling space offset field onto each spatial sampling coordinate of the standard feature extraction kernel.

[0128] During feature extraction, a deformation attention kernel with the aforementioned coordinate offset is applied to resample and calculate the quantized latent space features. In terms of physical execution, this means that pixels in the quantized latent space features are no longer limited to their original fixed coordinate positions for data extraction. Instead, they undergo geometric distortion and bilinear coordinate interpolation calculations along a vector path specified by the two-dimensional sampling space offset field, gradually moving towards the realistic and sharp edge pixels generated by the diffusion network in the texture enhancement features through a creeping traction and non-rigid movement. By performing this non-rigid movement calculation, the system eliminates the spatial misalignment bias caused by fixed-grid quantization at the spatial scale.

[0129] Step 605: Calculate the pixel-wise activation weights in the aligned feature space using a multi-channel group multiplication mechanism. After completing the deformation alignment calculation in the structural dimension, the system performs channel-level and pixel-level signal energy allocation on the dual-stream features. The system then concatenates and combines the aligned quantized latent space features and texture enhancement features along the channel dimension.

[0130] The system utilizes a feature evaluation network with a grouped convolutional structure to compute dependencies within the channel dimension. The system performs a nonlinear mapping operation on the output tensor of the feature evaluation network using an activation function, constraining its value within a closed interval of 0 to 1. Specifically, the activation function can be implemented using the sigmoid function. This nonlinear mapping operation generates a spatial matrix composed of multi-channel values ​​at each coordinate node of the feature map, i.e., the pixel-wise activation weights. These activation weights reflect the proportion of information contribution of each of the two input features to the reconstruction result at a specific channel at a specific spatial location.

[0131] Step 606: Using the activation weights, the aligned quantized latent space features and the texture enhancement features are weighted and synthesized to obtain a fused feature representation.

[0132] After obtaining the activation weights, element-wise matrix multiplication is performed based on these pixel-by-pixel activation weights. The element-wise product of the first channel subset of the activation weights and the aligned quantized latent space features is calculated, and the element-wise product of the second channel subset of the activation weights and the texture enhancement features is also calculated.

[0133] The system performs pixel-by-pixel additive synthesis on the result of the product of the two matrices. After the deformation alignment and adaptive weighted synthesis operations, the two sets of prior features from different parsing paths are tightly coupled and computed, outputting a continuous feature tensor that eliminates mesh artifact bias and preserves generation details, thus obtaining the fused feature representation. This optimized feature representation is then input into the subsequent decoding network to generate the final physical pixel image.

[0134] As a basic alternative implementation, if the current system's computing hardware unit does not support complex dynamic deformation sampling kernel floating-point operations, a conventional parallel feature weighted fusion scheme based on a multilayer perceptron can be used as a fallback.

[0135] Example 7 describes a three-stage progressive training strategy, cascaded residual codebook orthogonal regularization constraints, and a method for constructing a spatially heterogeneous bi-objective training loss for offline construction of an image compression and reconstruction network.

[0136] Step 701, Phase 1: Train the vector quantization encoder, concatenated residual codebook and decoder to minimize the reconstruction error and perceptual loss between the reconstructed image and the original image; When constructing a high-fidelity image compression and reconstruction model, the system needs to establish a fundamental extraction capability for discrete latent space representation. In this stage, the system initializes the network weights of the vector quantization encoder, the concatenated residual codebook, and the decoder. The system acquires a large number of training sample images, which can be obtained from publicly available image datasets, such as ImageNet, COCO, and DIV2K, which contain large-scale image datasets with diverse scenes and textures, or obtained through conventional image acquisition methods in this field. The specific scale of the training data can be adaptively adjusted according to the model capacity and target performance. This data is input into the vector quantization encoder to extract continuous raw, unquantized latent features, which are then discretized and mapped through the concatenated residual codebook. Finally, the decoder outputs the reconstructed image. The system calculates the difference data between the reconstructed image and the original training sample images to construct a joint loss function. This joint loss function includes a reconstruction error loss calculated using numerical absolute bias, a perceptual loss based on the visual geometric group network feature hierarchy extraction, a discriminator adversarial loss calculated based on a generative adversarial network architecture, and a commitment loss and codebook update loss used to constrain the encoder output to be closer to the codeword vector. The system uses the backpropagation algorithm to update the weight parameters of each network module until the joint loss function converges, thereby establishing the basic image reconstruction benchmark.

[0137] Step 702: In the first stage, an orthogonal regularization constraint is applied to the codebook matrix of each level in the concatenated residual codebook to promote the distribution of codewords at each level in mutually orthogonal subspaces and reduce inter-level redundancy.

[0138] To improve the feature representation efficiency of the cascaded residual codebook and avoid extracting highly similar repetitive features from quantization dictionaries at different levels, the system introduces an additional orthogonal regularization penalty term for the codebook matrix in the joint loss function of stage one. The formula for this calculation process is as follows: L_ortho=Sum_k(Norm_F(C k T × C k - I) 2 ); Where L_ortho is the orthogonal regularization constraint loss, Sum_k() is the mathematical summation function for all cascaded level indices k, Norm_F() is the function for calculating the Frobenius norm of the matrix, and C k Let C be the k-th level codebook matrix. k TLet I be the transpose of the k-th level codebook matrix, and let I be the identity matrix with the same feature dimension as the product of the codebook matrix. By minimizing this constraint loss during each weight parameter update, the system forcibly pushes out the feature basis vectors of different levels in the high-dimensional mathematical space, ensuring that the second-level refined dictionary and the third-level large-capacity dictionary focus only on capturing residual information orthogonal to the features of the previous level, thereby reducing the redundancy of information encoding.

[0139] Step 703, Stage 2: Freeze the vector quantization encoder, the concatenated residual codebook and the decoder, and train the granularity allocation network for generating the multi-level granularity mask to minimize image distortion under the target bit rate constraint; After completing the learning of basic discrete representation capabilities, the system enters the training phase of a dynamic resource allocation strategy. During this phase, the system stops calculating the parameter gradients of the vector quantization encoder, the concatenated residual codebook, and the decoder; that is, it performs a network freeze operation, only backpropagating the gradients to an independently constructed granular allocation network. This granular allocation network includes a statistical calculation module for extracting image patch entropy, a saliency extraction sub-network based on a feature pyramid architecture, and a comprehensive score calculation layer based on the exponential gating function.

[0140] The granularity allocation network calculates discrete, multi-level granularity masks based on the input training sample images. The system calculates an allocation loss function for this stage, including a distortion term and a bitrate penalty term. The distortion term evaluates the degree of structural distortion in the reconstructed image produced under the current granularity allocation strategy. The bitrate penalty term calculates the actual estimated bitrate by summing the number of blocks at each granularity level with the corresponding codebook index bit width, and then calculates the difference between the actual estimated bitrate and the user-defined target bitrate. When the actual estimated bitrate is greater than the target bitrate, the system calculates the square of the difference as the penalty value; when the actual estimated bitrate is less than or equal to the target bitrate, the penalty value is output as 0.

[0141] Since the output of the multi-level granularity mask is a discrete integer, the system uses a differentiable relaxation technique to calculate the approximate partial derivative of this discrete allocation process in the backpropagation calculation, so that the gradient can be successfully backpropagated to the granularity allocation network, prompting it to learn to output the optimal spatial granularity combination strategy under the physical constraint of a given transmission bandwidth limit.

[0142] Step 704, Stage 3: Freeze the aforementioned network and train the denoising network to learn the residual denoising mapping of the quantized latent space features to the unquantized latent space features output by the encoder.

[0143] After optimizing the granularity allocation network logic, the system initiates an independent optimization process for the latent diffusion enhancement module. The system simultaneously freezes the weight parameters of the vector quantization codec and the granularity allocation network. Using the frozen network modules, the system extracts the original unquantized latent features of the training sample images as a clear data distribution approximation target, and extracts the quantized latent space features reconstructed through a lookup table as the baseline structural condition. The system injects Gaussian noise into the original unquantized latent features and concatenates the noisy features with the baseline structural condition and the multi-level granularity mask along the channel dimension, inputting them into the denoising network to be optimized. The system calculates the deviation between the residual noise value predicted by the denoising network and the actual injected noise to update the denoising network weights.

[0144] Step 705: Assign fidelity weights and perceptual weights to each spatial location according to the multi-level granularity mask; wherein the fidelity weights are positively correlated with the granularity level, and the perceptual weights are negatively correlated with the granularity level. The pixel-level mean square error loss is calculated based on the deviation between the predicted noise and the actual injected noise of the denoising network, and the local perceptual loss is calculated based on the difference between the denoising result and the unquantized latent space features. In conventional training of denoising networks for image compression tasks, a globally uniform prediction error objective can lead to a physical conflict between the network's optimization direction and the naturalness of generated textures. To resolve this conflict, the system calculates spatially heterogeneous loss distribution weight parameters based on multi-level granularity masks. The system extracts the granularity level label values ​​corresponding to each spatial coordinate and performs linear interpolation calculations.

[0145] Specifically, the system divides the granularity level label value of the current coordinate point by a preset maximum level label constant, outputting the corresponding fidelity weight; the system then subtracts this fidelity weight from the constant 1, outputting the corresponding perceptual weight. Since the granularity level label value set consists of integer constants containing 0, 1, and 2, when the extracted granularity level is 2, the system outputs a fidelity weight of 1 and a perceptual weight of 0; when the extracted granularity level is 0, the system outputs a fidelity weight of 0 and a perceptual weight of 1. This numerical mapping process spatially binds the reconstruction accuracy in physical features with the optimization direction of subsequent calculations.

[0146] Step 706: Use the fidelity weights to weight the pixel-level mean square error loss and use the perceptual weights to weight the local perceptual loss to construct the total training loss function.

[0147] After obtaining the weight distribution matrix of the entire map coordinate dimensions, the system merges the above values ​​at the loss assessment calculation node.

[0148] L_dual=E_math[w_f(x,y)*L_mse(x,y)+w_p(x,y)*L_lpips(x,y)]; Where L_dual is the constructed total training loss function, E_math[] is the mathematical expectation calculation operation, w_f(x,y) is the fidelity weight corresponding to spatial coordinates (x, y), L_mse(x, y) is the pixel-level mean squared error loss calculated at spatial coordinates (x, y), w_p(x, y) is the perceptual weight corresponding to spatial coordinates (x, y), and L_lpips(x, y) is the local perceptual loss calculated at spatial coordinates (x, y).

[0149] Under the constraint of spatially weighted dual-objective loss with granular conditionalization, the denoising network performs heterogeneous parameter optimization schemes for different spatial features during the same forward computation and backward gradient propagation process. Specifically, the loss gradient parameters for fine-grained regions mainly originate from pixel-level mean square error loss, forcing the network to accurately deduce denoising details; while the loss gradient parameters for coarse-grained regions mainly originate from perceptual loss, driving the network to deduce and synthesize texture elements that conform to the distribution patterns of human vision, thus resolving the computational compromise problem caused by a single global loss objective.

[0150] In some alternative implementations, in order to reduce the computational time overhead of the third-stage bi-objective loss in the model training loop, the system can adjust the weight interpolation transition calculation method for the medium-granularity region in the multi-level granularity mask.

[0151] Example 8 describes the engineering implementation process of the high-fidelity image compression and reconstruction method in multiple actual physical hardware and business flow environments.

[0152] Step 801: Obtain the image to be compressed; In specific physical systems, the method of acquiring the image to be compressed is determined by the hardware front-end device. In mobile communication and interactive media applications, the image to be compressed is specifically a real-time interactive screen or a continuous image sequence captured by a mobile terminal camera. In space information and Earth observation applications, the image to be compressed is specifically a high-resolution remote sensing image matrix acquired by a multispectral sensor mounted on a low-Earth orbit satellite or aircraft. In industrial vision and security monitoring applications, the image to be compressed is specifically extracted frames from a continuous video sequence captured by a high-definition industrial camera deployed at a production line quality inspection station or a bandwidth-constrained monitoring node. The system's underlying driver reads the raw two-dimensional pixel arrays generated by the aforementioned different hardware sensors and writes them into the device's computing memory.

[0153] Step 802: Determine the information density and visual saliency of each region in the image to be compressed, and generate a multi-level granularity mask accordingly. The system invokes a granular evaluation model deployed in the front-end device's computing unit to perform spatial region segmentation on images of different physical scenes. In mobile communication applications, regions with high information density and visual saliency for face recognition or text are marked as fine-grained; solid-color backgrounds or out-of-focus blurred areas are marked as coarse-grained. In spatial information Earth observation applications, regions with complex structures such as urban building clusters, transportation networks, and military facilities receive high scores and are marked as fine-grained or medium-grained; regions such as oceans, deserts, and homogeneous forests are marked as coarse-grained. In industrial visual inspection applications, the system marks the coordinates of key features such as minor scratches on product surfaces, subtle defects in components, and instrument readings as fine-grained, while marking normal and smooth product shell areas as coarse-grained.

[0154] Step 803: Combine multi-level granularity masking to perform vector quantization encoding on the image to be compressed to obtain quantization latent space features; Within the hardware computing unit of the front-end device, the system performs cascaded quantization calculations and data stream encapsulation controlled by multi-level granularity masks. When the mobile communication network is under bandwidth constraints, the system dynamically adjusts the granularity ratio parameter to reduce the total proportion of fine-grained regions and performs non-uniform bitstream encapsulation calculations, ensuring that the amount of data occupied after full-image compression is within a set range of 0.05 bits per pixel to 0.1 bits per pixel.

[0155] Step 804: Guided by the multi-level granularity mask, perform latent diffusion enhancement processing on the quantized latent space features using a pre-trained denoising network to obtain texture enhancement features; wherein, the spatial regions corresponding to different granularity levels have heterogeneous denoising scheduling strategies when performing latent diffusion enhancement processing. After transmission through the communication channel, the cloud server or the receiving baseband processor parses the received non-uniform data stream, outputting quantized latent space features and multi-level granular masks. The cloud server has a built-in computing array with a graphics processing unit, running a pre-trained denoising network. During latent diffusion enhancement processing, heterogeneous computing power scheduling is performed based on spatial coordinates. For coarse-grained background regions in heavily compressed images transmitted from mobile terminals, the system allocates higher initial noise and longer inference time steps to calculate and synthesize complementary smooth textures; for scratch defect coordinate regions transmitted from industrial cameras, the system allocates extremely low initial noise and performs early freeze calculations to protect the geometric features of physical defects from being disrupted by generative computation. Through this heterogeneous computation, the system completes spatial heterogeneous data compensation for feature tensors during the cloud decompression process.

[0156] Step 805: Combine the multi-level granularity mask to perform spatial alignment fusion of the texture enhancement features and the quantization latent space features, and reconstruct the image based on the fusion result.

[0157] The receiving end computing unit performs spatial alignment and fusion calculations on the dual-stream features and uses the decoder to output a reconstructed two-dimensional pixel array. In spatial information Earth observation applications, the feature matrix extracted from the reconstructed remote sensing imagery in fine-grained areas such as building edges meets the pixel coordinate accuracy requirements of geographic information system analysis, environmental monitoring, and resource exploration applications. In industrial visual inspection applications, the reconstructed image retains the geometric morphological features of minute scratches and component defects. When the reconstructed image matrix is ​​input into the backend automated quality inspection and fault diagnosis model, the improved reconstruction fidelity of core defect areas helps to improve the accuracy of backend defect identification.

[0158] Furthermore, in an alternative implementation, the system can compile and encapsulate all the above-described method steps into a dynamic link library or hardware description language code. For cloud server environments with sufficient computing power, the system deploys each computational step in a graphics processing unit array to perform floating-point operations using matrix acceleration.

Claims

1. An image compression and reconstruction method based on granularity and diffusion model optimization, characterized in that, include: Obtain the image to be compressed; Determine the information density and visual saliency of each region in the image to be compressed, and generate multi-level granularity masks accordingly; The image to be compressed is vector quantized and encoded by combining multi-level granularity masks to obtain quantized latent space features; Guided by the multi-level granularity mask, a pre-trained denoising network is used to perform latent diffusion enhancement processing on the quantized latent space features to obtain texture enhancement features; Among them, the spatial regions corresponding to different granularity levels have heterogeneous denoising scheduling strategies when performing latent diffusion enhancement processing; By combining the multi-level granularity mask, the texture enhancement features and the quantization latent space features are spatially aligned and fused, and the image is reconstructed based on the fusion result.

2. The method according to claim 1, characterized in that, The generation of multi-level granularity masks includes: For each image patch in the compressed image, extract the unstructured information entropy and visual saliency score separately; The unstructured information entropy and the visual saliency score are mapped to preset dimensionless intervals to obtain normalized information entropy and normalized saliency score. The normalized information entropy and the normalized saliency score are nonlinearly weighted using a preset exponential gating function to obtain the comprehensive score for each image patch. Based on the comprehensive score and the preset granularity ratio parameter, the granularity level corresponding to each image block is determined, and a multi-level granularity mask is generated.

3. The method according to claim 1, characterized in that, The obtained quantized latent space features include: Provide a pre-built concatenated residual codebook, which contains multiple quantization dictionaries nested in a hierarchical manner; A waterfall-style retrieval is performed on the latent space features of the image to be compressed, the reconstruction residual after quantization at the current level is calculated, and the reconstruction residual is determined to enter the next level of quantization dictionary for iterative retrieval based on the granularity level indicated by the multi-level granularity mask. The codebook indexes obtained from each level of retrieval are combined to obtain the quantized latent space features.

4. The method according to claim 1, characterized in that, The heterogeneous denoising scheduling strategy includes: Based on the granularity level indicated by the multi-level granularity mask, determine the initial noise intensity and target denoising depth corresponding to each spatial location; The granularity level is negatively correlated with the initial noise intensity and the target denoising depth, so that image regions with lower quantization accuracy can obtain higher initial noise and a deeper number of denoising iterations.

5. The method according to claim 4, characterized in that, The execution of the latent diffusion enhancement process includes: Using the quantized latent space features as anchor points, corresponding random noise is injected into the latent space according to the initial noise intensity to obtain the initial noise latent features of spatial heterogeneity. Using the pre-trained denoising network, a multi-step reverse denoising iteration is performed on the initial noise latent features.

6. The method according to claim 1, characterized in that, After obtaining the texture enhancement features, the following is also included: Calculate the gradient magnitude of the multi-level granularity mask, and determine the cross-granularity boundary band based on the gradient magnitude; In the texture enhancement feature, local feature harmonic convolution is used to smooth the features located within the cross-granularity boundary band; The mixing intensity of the smoothing process is adaptively determined by the gradient magnitude, which is used to eliminate feature discontinuities caused by different denoising scheduling strategies.

7. The method according to claim 1, characterized in that, The spatial alignment and fusion includes: A gradient boundary matrix is ​​generated based on the local level transitions of the multi-level granularity mask. Based on the gradient boundary matrix, a learned two-dimensional sampling space offset field is generated at the overlapping edge of the texture enhancement feature and the quantization latent space feature. The two-dimensional sampling space offset field is used to guide the quantization latent space features to be aligned with the edge structure of the texture enhancement features.

8. The method according to claim 1, characterized in that, The denoising network is pre-trained in the following manner: Phase 1: Training the vector quantization codec to minimize the reconstruction error and perceptual loss between the reconstructed image and the original image; Phase 2: Freeze the vector quantization codec and train the granularity allocation network to minimize image distortion under the target bitrate constraint; Phase 3: Freeze the aforementioned network and train the denoising network to learn the residual denoising mapping from quantized features to original features.

9. The method according to claim 8, characterized in that, In stage three, training is performed using a granular conditional spatially weighted bi-objective loss, specifically including: According to the multi-level granularity mask, a fidelity weight and a perceptual weight are assigned to each spatial location; wherein, the fidelity weight is positively correlated with the granularity level, and the perceptual weight is negatively correlated with the granularity level; The pixel-level mean square error loss is weighted using the fidelity weights, and the local perceptual loss is weighted using the perceptual weights to construct the total training loss function.

10. The method according to claim 3, characterized in that, The execution of the waterfall-style search includes: All regional features are retrieved at the first level using the first-level coarse-grained codebook, and the first-level reconstruction mapping residual map is calculated. Based on the multi-level granularity mask, the first-level reconstruction mapping residual map corresponding to the medium and fine granularity regions is input into the second-level refinement dictionary for two-level retrieval, and the second-level reconstruction mapping residual map is calculated. The residual map of the second-level reconstruction mapping corresponding to the fine-grained region is input into the third-level large-capacity dictionary for three-level retrieval.

Images