One-step diffusion-based generative image compression decoding method, model construction method and system

Abstract

The application discloses a generative image compression decoding method and system based on one-step diffusion, and a model construction method. The method comprises the following steps: obtaining an image compression code stream and performing entropy decoding analysis to obtain a compressed latent variable; inputting the compressed latent variable into a preset latent variable adaptive network to perform dimension mapping and feature space alignment, and generating multi-scale conditional adaptive features; obtaining initial Gaussian random noise; synchronously inputting the multi-scale conditional adaptive features and the initial Gaussian random noise into a one-step diffusion generation network, injecting the conditional adaptive features of corresponding scales into multiple scale feature levels of the network respectively, and performing single forward propagation generation to obtain a reconstructed latent representation; and performing inverse transformation decoding on the reconstructed latent representation, and outputting a reconstructed image. The application reduces the dependence of traditional generative compression decoding on multi-step iterative sampling, and improves the subjective perception quality of image reconstruction under extremely low bandwidth and low decoding delay.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a generative image compression and decoding method, model construction method, and system. Background Technology

[0002] With the explosive growth in the scale of visual content, achieving high-fidelity transmission of image data in environments with extremely low available bandwidth has become a major technical challenge in the fields of communication and storage. Traditional image encoding and decoding techniques based on transform and prediction are prone to significant truncation of high-frequency features when faced with extremely low bitrate constraints, resulting in severe block artifacts and texture blurring in the reconstructed image. Therefore, exploring how to recover the physical structure and visual semantic distribution of an image to the greatest extent possible under extremely limited bitrates is of great significance for improving the fidelity of image transmission in harsh channel environments in terms of underlying data processing and information reconstruction.

[0003] In current cutting-edge research on image decoding, end-to-end learning-based compression networks optimize rate-distortion performance through data-driven approaches. However, because they primarily rely on pixel-level mean square error reconstruction constraints, these methods tend to output overly smoothed hidden layer representations at extremely low bitrates, making it difficult to synthesize sharp edges and realistic visual textures. To alleviate this phenomenon, generative models based on multi-step iterative denoising are gradually being introduced into the compression decoding process, utilizing their large number of parameters and generative priors to reshape the perceptual quality of images. However, such decoding techniques that rely on deep priors typically require performing dozens or even hundreds of Markov chain inverse inference calculations, resulting in a multi-fold increase in the end-side decoding time for a single image. Furthermore, when faced with perturbations of varying intensities of bit distortion or random initialization noise, existing generative pipelines are prone to structural misalignment or artifacts, causing the generated pixels to deviate from the original data distribution, severely limiting generalization ability and decoding robustness.

[0004] In summary, existing high-perceptual-quality image compression and decoding schemes generally suffer from a contradiction between reconstruction fidelity and edge inference timeliness. How to ensure stable and robust reconstruction of the high-level semantic structure and low-level texture details of images even at extremely low bitrates, while reducing hardware computational overhead and inference latency, is a common technical challenge that urgently needs to be solved in the fields of low-bandwidth image communication and generative encoding and decoding. Therefore, it is necessary to study an image compression and decoding method that can balance extremely low computational latency with high-quality perceptual reconstruction. Summary of the Invention

[0005] The purpose of this invention is to provide a generative image compression and decoding method, model building method, and system based on one-step diffusion, in order to solve the aforementioned problems in the prior art.

[0006] Technical solution: A generative image compression and decoding method based on one-step diffusion, comprising: Obtain the compressed image bitstream, perform entropy decoding and parsing on the compressed image bitstream to obtain the compression latent variables; The compressed latent variables are input into a pre-defined latent variable adaptation network for dimension mapping and feature space alignment to generate multi-scale conditional adaptation features. Obtain initial Gaussian random noise of a preset dimension. The initial Gaussian random noise is a reproducible noise tensor, generated by a preset fixed seed, or using a preset noise tensor, or determined by the bitstream field. The conditional adaptation features at multiple scales and the initial Gaussian random noise are input into the one-step diffusion generation network. The corresponding scale conditional adaptation features are injected into the multiple scale feature levels of the one-step diffusion generation network and generated by a single forward propagation to obtain the reconstructed latent representation. The reconstructed latent representation is decoded by inverse transform, and the reconstructed image is output.

[0007] According to one aspect of this application, compressed latent variables are input into a pre-defined latent variable adaptation network for dimensionality mapping and feature space alignment to generate multi-scale conditional adaptation features, including: The compressed latent variables are reduced in dimensionality by using a pre-constructed first convolutional layer to obtain dimensionality-reduced features. Spatial mapping features of the dimensionality reduction features are extracted using a pre-constructed second convolutional layer to obtain spatial modulation features; By using a pre-constructed third convolutional layer, the number of channels of the spatial modulation features is mapped to a preset target dimension to obtain the basic adaptation features; Multi-scale resolution interpolation is performed on the basic adaptation features to generate multi-scale conditional adaptation features.

[0008] According to one aspect of this application, the one-step diffusion generation network has an encoder-decoder architecture, and multiple scale feature levels include sequentially connected downsampling stage feature maps, bottleneck layer feature maps, and upsampling stage feature maps. Conditional adaptation features of corresponding scales are injected into multiple scale feature layers of the one-step diffusion generation network, including: The conditional adaptation features of the corresponding scale are injected into the feature maps of the downsampling stage and the upsampling stage respectively to construct generation constraints for local texture and high-frequency detail information. Conditional adaptation features of the corresponding scale are injected into the bottleneck layer feature map to construct generation constraints for global semantic and structural consistency.

[0009] According to one aspect of this application, conditional adaptation features of corresponding scales are injected into multiple scale feature layers of a one-step diffusion generation network, including: By using a cross-attention mechanism, the conditional adaptation features of the corresponding scale are injected into the current generated features at the corresponding scale level, thereby realizing information interaction between the conditional adaptation features and the current generated features.

[0010] According to one aspect of this application, a cross-attention mechanism is used to inject conditionally adapted features of the corresponding scale into the currently generated features at the corresponding scale level, including: Map the currently generated features to the query vector; The conditional adaptation features at the corresponding scales are mapped to key vectors and value vectors, respectively. Based on the similarity matching results between the query vector and the key vector, the value vector is weighted and summed to obtain the feature modulation result; Update the currently generated features using the feature modulation results.

[0011] According to one aspect of this application, the cross-attention mechanism employs a multi-head attention architecture; In this multi-head attention architecture, the number of attention heads and the total feature dimension after mapping are both preset fixed values ​​to ensure that the feature dimensions of the query vector, key vector, and value vector remain consistent.

[0012] According to one aspect of this application, during the process of injecting conditional adaptation features of corresponding scales into multiple scale feature layers of a one-step diffusion generation network and performing a single forward propagation generation, no probability-based zeroing operation is performed on the conditional adaptation features, so as to maintain the data integrity of the conditional adaptation features that serve as generation guidance conditions.

[0013] According to one aspect of this application, when the one-step diffusion generation network performs the single forward propagation, it also receives the time step embedding corresponding to the preset diffusion time step as an auxiliary input condition; the preset diffusion time step is the maximum diffusion time step T in the range of diffusion time steps used by the pre-trained diffusion model.

[0014] According to one aspect of this application, the preset field of the image compression bitstream further carries color statistics, the color statistics including the mean and standard deviation of each color channel of the original image after quantization encoding; after performing inverse transform decoding on the reconstructed latent representation and outputting the reconstructed image, the method further includes: parsing the color statistics from the preset field of the image compression bitstream; and performing distribution renormalization processing on the reconstructed image based on the color statistics to obtain a corrected reconstructed image.

[0015] According to one aspect of this application, before obtaining the image compressed bitstream, performing entropy decoding on the image compressed bitstream, and obtaining the compressed latent variables, a model pre-construction stage is included, which includes: Obtain training image samples; The compressed latent variables corresponding to the training image samples are extracted based on the preset image coding components and used as training conditions. The initial latent variable fitting network and the initial one-step diffusion generation network are fine-tuned end-to-end using a joint loss function to obtain the preset latent variable fitting network and one-step diffusion generation network.

[0016] According to one aspect of this application, an end-to-end joint fine-tuning of the initial latent variable fitting network and the initial one-step diffusion generation network is performed using a joint loss function, including: Calculate the output consistency constraint term for different initial noise perturbations; Calculate the supervised reconstruction loss term for the generated quality; A joint loss function is constructed based on the output consistency constraint term and the supervised reconstruction loss term, and the network parameters are iteratively updated.

[0017] According to one aspect of this application, the output consistency constraint term is calculated for different initial noise perturbations, including: While maintaining the same training conditions, input the first random noise and the second random noise into the initial one-step diffusion generator network, respectively; The first single-step prediction result of the initial one-step diffusion generation network based on the first random noise output and the second single-step prediction result based on the second random noise output are obtained respectively. Calculate the mean square error between the first single-step prediction result and the second single-step prediction result, and use it as the output consistency constraint term.

[0018] According to one aspect of this application, the supervised reconstruction loss term for generated quality is calculated, including: Using a pre-built multi-step diffusion decoder as the teacher model, multi-step denoising is performed on the training image samples to generate multi-step denoised pseudo-real values. The approximation error between the initial one-step diffusion generation network's single-step prediction based on training conditions and the multi-step denoising pseudo-true value is calculated and used as the supervised reconstruction loss term.

[0019] Specifically, a pre-built multi-step diffusion decoder is used as the teacher model, and the training conditions are used as conditions to guide multi-step denoising processing to generate multi-step denoised pseudo-real values. Obtain the single-step prediction result obtained by the initial one-step diffusion generation network through a single forward propagation based on the training conditions; The single-step prediction result and the multi-step denoising pseudo-true value are spatially and dimensionally aligned, and the approximation error between them is calculated. The approximation error is used as the supervised reconstruction loss term.

[0020] According to one aspect of this application, the supervised reconstruction loss item is calculated using the following alternative method: Use training image samples directly as the target ground value; The single-step prediction result is decoded to the image space through inverse transformation, or the training image samples are transformed to the latent space through encoder. After aligning the space and dimension, the reconstruction error between the single-step prediction result and the target ground value is calculated as the supervised reconstruction loss term.

[0021] According to one aspect of this application, the process of end-to-end joint fine-tuning of the initial latent variable fitting network and the initial one-step diffusion generation network using a joint loss function further includes: The training conditions are zeroed out using a preset probability to enhance the generation robustness of the one-step diffusion generator network when faced with loss of truth in compressed code.

[0022] According to one aspect of this application, during the processing using the first convolutional layer, the second convolutional layer and the third convolutional layer, nonlinear feature processing is performed sequentially between adjacent convolutional layers using the GELU activation function and the LayerNorm normalization layer. Furthermore, the basic adaptation features are adjusted by multi-scale resolution interpolation. Specifically, a bilinear interpolation algorithm is used to align the spatial resolution of the basic adaptation features to the scale of the target feature map.

[0023] According to one aspect of this application, obtaining the image compressed bitstream specifically involves receiving the image compressed bitstream via a low-bandwidth wireless channel; and performing inverse transform decoding on the reconstructed latent representation on a neural network inference acceleration chip of a mobile terminal.

[0024] According to another aspect of this application, a generative image compression and decoding system based on one-step diffusion is also provided, comprising: The decoding and parsing module is used to obtain the image compressed bitstream, perform entropy decoding and parsing on the image compressed bitstream, and obtain the compression latent variables; The feature adaptation module is used to input compressed latent variables into a preset latent variable adaptation network for dimension mapping and feature space alignment, generating multi-scale conditional adaptation features. The noise acquisition module is used to acquire initial Gaussian random noise of a preset dimension; the initial Gaussian random noise is a reproducible noise tensor, which is generated by a preset fixed seed, uses a preset noise tensor, or is determined according to a preset field in the image compression bitstream.

[0025] The single-step generation module is used to input multi-scale conditional adaptation features and initial Gaussian random noise into the one-step diffusion generation network. The corresponding scale conditional adaptation features are injected into the multiple scale feature levels of the one-step diffusion generation network and generated by a single forward propagation to obtain the reconstructed latent representation. The inverse transform output module is used to perform inverse transform decoding on the reconstructed latent representation and output the reconstructed image.

[0026] A model construction method for one-step diffusion generative image compression and decoding includes: Obtain training image samples; Based on a preset image coding component, the compressed latent variables corresponding to the training image samples are extracted, and the compressed latent variables are used as training conditions. The training conditions are input into the initial latent variable adaptation network for dimension mapping and feature space alignment to generate multi-scale training condition adaptation features. Based on the multi-scale training condition adaptation features and initial Gaussian random noise, the initial one-step diffusion generation network is made to perform a single forward propagation to obtain a single-step prediction result; the output consistency constraint term is calculated for different initial noise perturbations. Calculate the supervised reconstruction loss term for the generated quality; A joint loss function is constructed based on the output consistency constraint term and the supervised reconstruction loss term; Based on the joint loss function, the initial latent variable adaptation network and the initial one-step diffusion generation network are jointly fine-tuned end-to-end to obtain the latent variable adaptation network and the one-step diffusion generation network for image compression and decoding.

[0027] According to one aspect of this application, the calculation of the supervised reconstruction loss term for the generated quality includes: Using a pre-built multi-step diffusion decoder as the teacher model, and guided by the training conditions, multi-step denoising is performed to generate multi-step denoised pseudo-real values. Obtain the single-step prediction result obtained by the initial one-step diffusion generation network through a single forward propagation based on the training conditions; Spatial and dimensional alignment are performed on the single-step prediction result and the multi-step denoised pseudo-true value, and the approximation error between the single-step prediction result and the multi-step denoised pseudo-true value is calculated. The approximation error is used as the supervised reconstruction loss term.

[0028] Beneficial effects By constructing a single forward propagation link that receives Gaussian random noise, the dependence on Markov chain-style multi-step denoising loops can be reduced, and the computational latency of image decoding can be reduced from the order of multi-step iterations to the order of single inference. This frees up the hardware computational overhead of the end-side device and helps to solve the technical problem of slow decoding caused by the reliance on multiple iterations in traditional diffusion models.

[0029] By introducing latent variable adaptation topology and cross-attention feature fusion algorithms, compressed information is mapped to conditional key values. Multi-scale concurrent injection is performed in the deep semantic nodes and shallow texture nodes of the generator network, so that the generated image can reconstruct local high-frequency edges while preserving the global image contour, thus reducing the defects of image texture blurring and structural distortion at extremely low bit rates.

[0030] To address the robustness deficiencies of single-step generation, which is susceptible to noise perturbations and bit errors, a noise-resistant output consistency constraint and a random conditional zeroing mechanism are deployed during the network fine-tuning stage to suppress the bias dependence of the inference network on the initial random state. Furthermore, to address the inherent color distribution drift problem of generated pixels, distribution renormalization is performed by extracting minimal overhead original feature statistics along the path, mapping the color matrix of the reconstructed pixels back to the original data distribution space. This improves the subjective perceptual quality of the reconstructed image under limited bandwidth channels and alleviates the overall color shift problem. Attached Figure Description

[0031] Figure 1 This is a flowchart of the generative image compression and decoding method based on one-step diffusion according to the present invention.

[0032] Figure 2 This is a flowchart of the process for generating multi-scale conditional adaptation features according to the present invention.

[0033] Figure 3 This is a flowchart illustrating how the present invention injects conditional adaptation features of the corresponding scale into the currently generated features at the corresponding scale level.

[0034] Figure 4 This is a flowchart of the pre-built model of the present invention. Detailed Implementation

[0035] The main workflow of this invention is as follows: the encoding end extracts features from the original image to obtain latent variables; the latent variables are quantized and entropy encoded based on the entropy model to obtain the image compressed bitstream; the decoding end performs entropy decoding to obtain the compressed latent variables; subsequently, latent variable adaptation, multi-scale conditional injection, one-step diffusion generation, and inverse transform decoding are performed.

[0036] Since the encoding and decoding processes are symmetrical, it is sufficient to describe only one of them. The decoding process is described in detail below.

[0037] Example 1: like Figure 1 As shown, this embodiment provides a generative image compression and decoding method based on one-step diffusion, which mainly includes the following steps: Step 101: Obtain the image compressed bitstream, perform entropy decoding and parsing on the image compressed bitstream to obtain the compression latent variables.

[0038] At the source of image encoding and transmission, the original image usually needs to be preprocessed by a learning compression framework to generate a hidden layer representation with compact characteristics, i.e., compressed latent variables.

[0039] Specifically, the source uses a pre-trained nonlinear transform encoder (e.g., a convolutional neural network) to downsample and extract features from the input raw RGB image data, and uses a specific rate-distortion optimization objective to transform it into a feature map with reduced spatial size and increased number of channels.

[0040] Furthermore, the feature map is transformed into a discrete representation through quantization and then subjected to lossless entropy encoding using a probabilistic estimation model, ultimately generating bitstream data suitable for transmission or storage in network channels. Based on this, the decoding end in this embodiment first acquires the image compressed bitstream. The image compressed bitstream is generated by the encoding end based on image encoding components, quantization processing, and entropy encoding processing, and carries probabilistic model information or corresponding analytical information for recovering the compression latent variables.

[0041] The decoding end uses an entropy decoder that matches the source end to parse and reconstruct the received bitstream based on an arithmetic decoding probability model. Through this reverse operation, the bitstream is deconstructed into a feature matrix in tensor form, i.e., compressed latent variables.

[0042] The compressed latent variables, in the form of an implicit multidimensional matrix, carry the texture, color distribution, and core structured semantic information of the original image, forming the data basis for the subsequent conditional reconstruction by the diffusion generation network.

[0043] For example, in applications involving image processing at standard resolutions, the parsed compressed latent variables are typically represented as three-dimensional feature tensors with a shape of H*W*192, where H and W represent the spatial height and width dimensions after downsampling, respectively, and 192 represents the number of feature channels contained in the latent variable.

[0044] It is important to note that, conventionally, two-dimensional spatial graphs are called feature graphs. Three-dimensional and higher-dimensional graphs are called feature tensors or tensor matrices.

[0045] Step 102: Input the compressed latent variables into the preset latent variable adaptation network for dimension mapping and feature space alignment to generate multi-scale conditional adaptation features.

[0046] Because the learning-based compression framework used in the front end and the diffusion-generative network used in the back end differ in their design goals and architectural evolution paths, the compressed latent variables obtained by direct parsing are often difficult for the diffusion model to directly utilize in terms of channel number and spatial representation logic.

[0047] To eliminate the barriers between these heterogeneous feature spaces, the system introduces a pre-defined latent variable adaptation network between the latent variable decoding and diffusion generation stages.

[0048] Specifically, the latent variable adaptation network first receives compressed latent variables through a series of cascaded linear and nonlinear transformation layers, and then performs channel dimension mapping adjustment on them through cascaded convolutional layers. That is, it first performs channel dimensionality reduction to remove redundant channels, and then extracts spatial mapping features and maps the number of channels to a preset target dimension.

[0049] Furthermore, since the diffusion generation network adopts a multi-scale hierarchical topology that includes multiple upsampling and downsampling, the latent variable adaptation network also needs to scale the spatial dimension of the feature matrix after channel mapping using a resolution adjustment algorithm based on the receiver port size of each level within the diffusion network.

[0050] Through the above operations, the latent variable adaptation network outputs a set of conditional adaptation features that match the layers of the diffusion network in terms of both channel number and spatial size. This set of multi-scale feature tensors is not only aligned mathematically, but also bridged in terms of deep semantic logic, enabling the diffusion generator to read the originally highly compact compressed information in an easily understandable and integrated form.

[0051] It should be noted that the compressed latent variables and the denoised latent representations within the one-step diffusion generation network reside in different feature spaces: the compressed latent variables are only transformed into multi-scale conditional adaptation features through the latent variable adaptation network, participating in generation guidance in the form of cross-attention conditions, and do not directly participate in noise evolution.

[0052] In some alternative implementations, the latent variable adaptation network can be constructed using a stacked convolution kernel from shallow to deep, and combined with traditional resampling operators such as bilinear interpolation to achieve scale variation, so as to balance mapping accuracy and lightweight computational resource overhead.

[0053] Step 103: Obtain initial Gaussian random noise of a preset dimension. The initial Gaussian random noise is a reproducible noise tensor, generated from a preset fixed seed, using a preset noise tensor, or determined by the bitstream field. In the standard iterative sampling chain of traditional diffusion models, the generation process begins with a clean noise distribution and gradually approximates the real data distribution through multiple rounds of inference by the denoising network. Within the decoding framework of this method, the generation paradigm starting from random perturbations is still used, but the aim is to traverse the entire denoising interval in a single inference.

[0054] Specifically, a pseudo-random number generation algorithm is used to sample a multidimensional random numerical matrix from a standard normal distribution. This random numerical matrix constitutes the initial Gaussian random noise, whose dimensions are constrained to match the shape of the basic feature tensor required by the input of the diffusion network. This initial Gaussian random noise represents the purely random state of the diffusion dynamics model at the maximum time step, and can be characterized, for example, as a four-dimensional data block with a shape of 1*4*64*64, where 1 is the batch dimension, 4 is the number of feature channels, and 64*64 is the spatial resolution. This operation establishes the initial starting point of the generative network.

[0055] When performing a single forward propagation, the one-step diffusion generation network receives a time step embedding fixed at the maximum diffusion time step T as an auxiliary input condition for the network. Since the one-step diffusion generation network is initialized with the weights of the pre-trained multi-step diffusion model, its time step embedding follows the time step semantics of the pre-trained model. Fixing the maximum input time step T can make the single-step output consistent with the initial state semantics of the complete denoised trajectory of the pre-trained model.

[0056] Step 104: Input the multi-scale conditional adaptation features and initial Gaussian random noise into the one-step diffusion generation network. Inject the corresponding scale conditional adaptation features into the multiple scale feature levels of the one-step diffusion generation network and perform a single forward propagation to generate the reconstructed latent representation.

[0057] After acquiring the conditional adaptation features containing structural and texture guidance information, as well as the initial Gaussian random noise that serves as the generation noise floor, both are synchronously transmitted to the processing pipeline of the core one-step diffusion generation network.

[0058] Furthermore, this one-step diffusion generation network abandons the traditional serial multi-step denoising loop during computation. Initial Gaussian random noise serves as the backbone input of the network, entering multi-level processing layers after passing through shallow extraction layers within the network. At different depth nodes of the network, i.e., multiple scale feature levels, the network reads conditional tensors of corresponding sizes from multi-scale conditional adaptation features based on the current channel and spatial state.

[0059] The conditional tensor is injected into the backbone data stream in real time through a specific feature fusion operator, thereby imposing targeted modulation and constraints on the intermediate feature maps evolved from random noise. The entire process requires only one complete feedforward network propagation computation.

[0060] Through multi-level feature permeation combining shallow and deep approaches, the one-step diffusion generative network can directly leap from a high-dimensional random space to a hidden image space with clear image meaning based on a single inference, outputting a denoised and reconstructed multi-dimensional feature set, i.e., a reconstructed latent representation. This single-step architecture reduces sensitivity to initial random noise perturbations while maintaining generative prior expressive power and perceptual quality.

[0061] In some alternative implementations, for real-time communication devices that are extremely sensitive to latency requirements, the internal weights of the one-step diffusion generation network can undergo intensive knowledge distillation compression during the pre-deployment phase to ensure that the number of floating-point operations in a single forward propagation process is controlled within the computational limit of the neural network inference acceleration chip. During the inference phase, pseudo-random noise generated from a preset fixed seed, a preset noise tensor, or pseudo-random noise determined by the bitstream field is used to enhance the reproducibility of the reconstruction results.

[0062] Step 105: Perform inverse transform decoding on the reconstructed latent representation and output the reconstructed image.

[0063] Although the reconstructed latent representation generated by the one-step diffusion network output has clear image topology and texture properties, it is still in a compressed high-dimensional latent space, which does not conform to the conventional observation format of the human visual system.

[0064] To complete the final image reconstruction, a pixel-level inverse transform decoding module based on a variational autoencoder (VAE) or similar architecture is employed to upsample and expand the received reconstructed latent representation and perform nonlinear color channel inversion. This inverse transform decoding process maps abstract hidden layer features back to the standard red, green, and blue three-component color space, recovering two-dimensional grid data corresponding to the resolution of the original input image, and visually approximating the original input image. The final output is a reconstructed image that can be directly rendered by a display device. Similar architectures include convolutional upsampling network architectures.

[0065] Example 2: The internal physical topology of the latent variable adaptation network and the one-step diffusion generation network is further explained.

[0066] like Figure 2 As shown, in one possible implementation, the following steps are included: Step 201: Use the pre-constructed first convolutional layer to perform channel dimensionality reduction on the compressed latent variables to obtain dimensionality-reduced features.

[0067] The compressed latent variables output by the front end typically contain redundant channel features. To reduce the computational cost of subsequent feature matching, a first convolutional layer consisting of 1*1 convolutional kernels is constructed. This layer performs a linear combination operation across channels on the input data. Specifically, for an initial tensor with 192 channels, the first convolutional layer compresses its channel count to 128, thereby outputting dimensionality-reduced features.

[0068] Furthermore, between adjacent convolutional layers, the GELU (Gaussian Error Linear Unit) activation function and the LayerNorm (layer normalization) normalization layer are sequentially used for nonlinear feature processing. By incorporating this activation and normalization mechanism at the output of the first convolutional layer, gradient vanishing is suppressed while stabilizing the mean and variance of the feature distribution, providing a standardized input basis for subsequent spatial feature extraction.

[0069] Step 202: Use the pre-constructed second convolutional layer to extract the spatial mapping features of the dimensionality reduction features to obtain the spatial modulation features.

[0070] After streamlining the channel dimensions, the network needs to capture pixel dependencies within the local space. A second convolutional layer with 3x3 kernels is configured, with a stride of 1 and a padding parameter of 1, to maintain numerical consistency between input and output in terms of spatial resolution.

[0071] The second convolutional layer performs sliding window computation on the two-dimensional spatial matrix of the reduced-dimensional features, aggregating texture and boundary information within adjacent receptive fields to generate spatial modulation features that reflect the local topological relationships of the image. After this operation, the aforementioned GELU activation function and LayerNorm normalization layer are called again to process the spatial modulation features, ensuring the continuity of nonlinear mapping capability in the network depth direction.

[0072] Step 203: The number of channels of the spatial modulation features is mapped to the preset target dimension using the pre-constructed third convolutional layer to obtain the basic adaptation features.

[0073] In order for compressed information to be seamlessly embedded into the diffusion generation network as a downstream module, its number of channels must be expanded to match the order of magnitude of the key-value receiving ports of the cross-attention mechanism within the diffusion generation network.

[0074] Another set of 1x1 third convolutional layers is used to perform channel upscaling on the spatial modulation features. For example, the number of channels is mapped from 128 to 512. The resulting base adaptation features achieve the preset target dimension of 512 in terms of channel dimension, thus unifying the data dimensions.

[0075] At this point, the basic adaptation features exist in the form of a three-dimensional tensor, and the number of its channels corresponds to the total dimension of the feature channels required for attention computation in the diffusion network. The above channel configuration can be adaptively adjusted according to actual network capacity requirements and hardware computing resources.

[0076] Step 204: Perform multi-scale resolution interpolation adjustment on the basic adaptation features to generate multi-scale conditional adaptation features.

[0077] Since the one-step diffusion generation network contains feature maps with different downsampling levels, the conditional features to be injected must correspond one-to-one with them in terms of spatial scale. Specifically, a bilinear interpolation algorithm is used to align the spatial resolution of the basic adaptation features to the scale of the target feature map.

[0078] This interpolation algorithm calculates a new pixel value based on a weighted average of four known pixels surrounding the target location. This allows the basic adaptation features to be scaled up or down to multiple given target sizes as needed, without changing the number of channels, resulting in a set of multi-scale conditional adaptation features covering various resolution matrices. In some optional implementations, interpolation coefficients can be dynamically calculated based on the scale parameters of different network layers to adapt to generative network models with asymmetric encoding / decoding structures.

[0079] Step 205: Inject the conditional adaptation features of the corresponding scale into the feature maps of the downsampling stage and the upsampling stage, respectively, to construct generation constraints for local texture and high-frequency detail information.

[0080] The one-step diffusion generation network in this method has an encoder-decoder architecture, and multiple scale feature layers include sequentially connected downsampling stage feature maps, bottleneck layer feature maps, and upsampling stage feature maps.

[0081] Specifically, the network employs a U-Net backbone comprising four levels of downsampling modules and four levels of upsampling modules. When processing the downsampling and upsampling feature maps located at the two ends of the network, these layers typically contain high-frequency detail information such as image edges and corners.

[0082] During a single forward propagation, conditional adaptation features corresponding to the spatial resolution are input into the aforementioned shallow network module. This operation limits the random divergence range of the generative network when reconstructing local pixel patterns by providing reference features of the same scale, guiding the model to recover the local texture features of the image.

[0083] Step 206: Inject the conditional adaptation features of the corresponding scale into the bottleneck layer feature map to construct generation constraints for global semantic and structural consistency.

[0084] The bottleneck layer feature map, located at the deepest part of the one-step diffusion generative network, has the smallest spatial resolution and the largest receptive field, and is mainly used to encode the global semantic layout and macroscopic structural features of the image. The feature matrix interpolated to the lowest resolution is extracted from the multi-scale conditional adaptation features and input into the bottleneck module for fusion calculation.

[0085] This process enables the diffusion generation model to obtain prior guidance on the overall layout of the target image during a single denoising inference stage, avoiding the structural collapse or object misalignment problems that often accompany single-step generation. By applying multi-scale constraints separately to the shallow and deep layers of the network in a decoupled design, the final output image maintains consistency in data distribution with the original bitstream in both macroscopic contours and microscopic details.

[0086] In other alternative implementations, the one-step diffusion generation network can also be implemented using a diffusion backbone network based on the Transformer architecture.

[0087] Example 3: After determining the network topology in the above embodiments, the specific calculation logic and inference state control of injecting condition-adaptive features into the currently generated features are further revealed.

[0088] In one possible implementation, the following steps are included: Step 301: The conditional adaptation features at the corresponding scale are injected into the current generated features at the corresponding scale level through a cross-attention mechanism to obtain the fused generated features; wherein, the current generated features are intermediate feature representations obtained after the initial Gaussian random noise is processed by the pre-sequence layer of a one-step diffusion generation network. This realizes the information interaction between the conditional adaptation features and the current generated features.

[0089] The goal is to handle the fusion of data streams originating from independent sources. The generated features are specifically represented as the hidden layer representation matrix maintained by a one-step diffusion generation network in processing layers of a specific depth. This hidden layer representation matrix evolves from initial Gaussian random noise through multiple rounds of nonlinear computation in preceding convolutional layers, carrying progressively evolving physical feature information. The cross-attention mechanism achieves fusion by constructing a correlation metric link between two different data domains.

[0090] This mechanism enables the generative network to adaptively extract compressed prior information with corresponding spatial relationships and semantic connections from the conditional adaptation features of external input when computing each spatial pixel in the current layer's feature map. In summary, establishing a content-based dynamic gated routing transforms the purely noise-driven unconditional generation process into a conditional generation process modulated by a specific compressed bitstream.

[0091] like Figure 3 As shown, in step 302, the currently generated features are mapped to query vectors; the conditional adaptation features of the corresponding scale are mapped to key vectors and value vectors respectively; based on the similarity matching results between query vectors and key vectors, the value vectors are weighted and summed to obtain feature modulation results; and the currently generated features are updated using feature modulation results.

[0092] To quantify the aforementioned information interaction process at the mathematical computation level, a set of independent linear mapping operators is defined. Specifically, a matrix multiplication mapping operation is performed on the currently generated features to generate a query vector. Simultaneously, two sets of parallel matrix multiplication mapping operations are performed on the conditionally adapted features at the corresponding scale to generate key vectors and value vectors, respectively.

[0093] Specifically, the projection operation relationship is as follows: Q=W q *F; K=W k *C; V=W v *C; In the formula, Q is the query vector, and W... q Here, F is the pre-trained query mapping weight matrix, F is the currently generated feature, K is the key vector, and W is the weight matrix. k Here, C is the pre-trained key-map weight matrix, C is the conditional fit feature at the corresponding scale, V is the value vector, and W is the weight matrix. v Map weight matrices to pre-trained values.

[0094] After obtaining the three mapping vectors mentioned above, the matrix dot product of the query vector and the key vector is calculated to measure the similarity matching result between the current state space of the generator network and the external compressed prior space in a specific semantic dimension. The standard scaled dot product cross-attention algorithm is used to perform Softmax feature normalization on the similarity matching result, and the normalized coefficient tensor is used to perform a weighted summation calculation on the value vector to output a feature matrix that incorporates relevant prior information, i.e., the feature modulation result.

[0095] In summary, an output linear mapping layer maps the number of channels in the feature modulation result back to the original number of channels in the currently generated features from a fixed total feature dimension. A residual connection operation is then performed, adding the mapped feature modulation result element-wise to the currently generated features, thereby updating and replacing the feature representation within the network.

[0096] Step 303: The cross-attention mechanism adopts a multi-head attention architecture; wherein the number of attention heads and the total feature dimension after mapping are preset fixed values ​​to ensure that the feature dimensions of the query vector, key vector and value vector are consistent.

[0097] To enable the network to concurrently extract feature associations from multiple independent representation subspaces, a cross-attention mechanism is instantiated as a multi-head attention architecture. During concurrent extraction computation, the total number of channels is equally divided into multiple discrete computation groups. For example, the preset number of attention heads is 8, and the total feature dimension after mapping is set to 512.

[0098] With this parameter configuration, each independent attention computation head is assigned a feature channel dimension of 64. Due to the adoption of a unified dimensionality reduction and mapping standard, the query vector, key vector, and value vector entering this computation component have their respective feature dimension values ​​aligned and kept equal before performing dot product and weighted summation operations.

[0099] The symmetry design at the dimensional level prevents dimensional incompatibility errors caused by different tensors when performing matrix multiplication, and controls the scale of floating-point operations in a single forward propagation at the hardware level. In some optional implementations, the total feature dimension can be adjusted according to the memory capacity configuration of the processing hardware, as long as the value after dividing by the number of attention heads is kept as a positive integer.

[0100] Step 304: During the process of injecting conditional adaptation features of the corresponding scale into the multiple scale feature layers of the one-step diffusion generation network and performing a single forward propagation generation, no probability-based zeroing operation is performed on the conditional adaptation features to maintain the data integrity of the conditional adaptation features that serve as the generation guidance conditions.

[0101] In conditionally guided model components, a dynamic feature discarding algorithm module is typically configured. This module relies on a random number generator to generate a Bernoulli distribution mask, which is then triggered with a set probability to overwrite some of the input guiding condition matrix values ​​with zeros. However, in the image compression and decoding stage, which this invention focuses on, the goal is to approximate the true reconstruction target based on the transmitted bitstream data. Therefore, during a single forward propagation generation task, the random zeroing function of the feature discarding module is disabled in the inference state.

[0102] It should be noted that this shutdown operation only applies to the online inference phase; during the offline training and fine-tuning phase, the training conditions are still randomly zeroed out according to step 403 with a preset probability to enhance the model's robustness to code slippage and bit errors. These two operations occur at different stages and do not conflict with each other. This instruction intercepts all probability-based zeroing operations on conditional adaptation features, which helps maintain the conditional adaptation features obtained from the decoder, including structural constraints and texture guidance matrices, allowing for full input into the one-step diffusion generation network without data truncation. Maintaining this inference state preserves the complete data picture of the prior guiding conditions, reducing structural reconstruction errors caused by missing conditions during single-step network inference computation.

[0103] Example 4: This embodiment provides a model construction method for one-step diffusion generative image compression and decoding, used to construct the latent variable adaptation network and one-step diffusion generative network employed in the aforementioned image compression and decoding method. For example... Figure 4 As shown, in one possible implementation, the following steps are included: Step 401: Obtain training image samples.

[0104] During the initialization phase of fine-tuning the model parameters, representative image tensors need to be retrieved from a pre-built image database as the training basis. For example, fixed-dimensional color image arrays can be batch-loaded from publicly available dimension-matched high-resolution image datasets (such as the DIV2K or Flickr2K datasets) as training image samples. This operation provides benchmark data objects for the network's forward propagation and error calculation.

[0105] Step 402: Extract the compressed latent variables corresponding to the training image samples based on the preset image coding components, and use them as training conditions.

[0106] Training image samples are input into the image encoding component, which is in a parameter-frozen or very low-learning-rate state, and nonlinear convolution and downsampling calculations are performed. The calculated output three-dimensional feature matrix is ​​defined as a compressed latent variable. In the fine-tuning loop, this matrix serves as a priori function to pass on the image semantics and topology to the diffusion generator and is established as a training condition for subsequent forward propagation of the network.

[0107] Before performing joint fine-tuning, the network weights of the base diffusion model, which has undergone multi-step denoising training on large-scale image data, are loaded into the initial one-step diffusion generation network as the starting point for fine-tuning; the weights of the initial latent variable adaptation network are set using a random initialization method. This initialization strategy enables the one-step diffusion generation network to have basic image generation capabilities at the beginning of the fine-tuning stage, reducing the convergence difficulty of joint training.

[0108] Furthermore, the training conditions are input into the initial latent variable adaptation network for dimensional mapping and feature space alignment to generate multi-scale training condition adaptation features; the multi-scale training condition adaptation features serve as condition guidance for the initial one-step diffusion generation network in subsequent forward propagation.

[0109] Step 403: Perform a feature zeroing operation on the training conditions with a preset probability to obtain perturbed training conditions; input the perturbed training conditions into the initial latent variable adaptation network for dimension mapping and feature space alignment to generate multi-scale training condition adaptation features; input the multi-scale training condition adaptation features and the sampled initial Gaussian random noise into the initial one-step diffusion generation network for a single forward propagation to enhance the generation robustness of the one-step diffusion generation network obtained after training when facing compression code loss truth.

[0110] Before each iteration, a random number generator is activated to produce a mask tensor that conforms to a Bernoulli distribution. This mask tensor is then used to perform conditional discarding calculations on the training conditions. For example, the preset probability can be limited to 10%. When the zeroing operation is triggered, the numerical matrix within the training conditions is covered by a tensor of all zeros. This perturbation mechanism blocks the overfitting tendency of the diffusion network to specific input channel data structures, enabling the model to perform reasonable pixel inferences based on the diffusion prior distribution within the network even when conditional features are incomplete or transmission channel errors occur, thus improving the stability of the network's inference state.

[0111] Step 404: Construct the joint loss function and perform end-to-end joint fine-tuning. The joint loss function consists of an output consistency constraint term for different initial noise perturbations and a supervised reconstruction loss term for the generated quality, including: The calculation of output consistency constraints for different initial noise disturbances includes steps 405 to 407: Step 405: Keeping the same training conditions, sample and obtain the first random noise and the second random noise respectively, and input the first random noise and the second random noise into the initial one-step diffusion generation network respectively.

[0112] Upon entering the output consistency calculation branch, the current training conditions are copied into memory to maintain the consistency of the reference matrix. The Gaussian random sampling operator is invoked to extract two independent four-dimensional noise tensors from the standard normal distribution, which are defined as the first random noise and the second random noise, respectively.

[0113] The copied training conditions are channel-correlated with the two independent noise tensors mentioned above to construct two sets of parallel input data pairs.

[0114] Step 406: Obtain the first single-step prediction result of the initial one-step diffusion generation network based on the first random noise output, and the second single-step prediction result based on the second random noise output.

[0115] The initial one-step diffusion generation network performs independent forward propagation on the two sets of parallel input data pairs. Under the same training conditions, the network's cross-attention mechanism performs feature mapping calculations on two different sets of initial Gaussian random noise, ultimately generating two sets of tensors of corresponding sizes at the output port. These two sets of tensors constitute the first and second single-step prediction results. The difference between these two sets of prediction tensors reflects the sensitivity of the current network parameter state to initial noise floor perturbations.

[0116] Step 407: Calculate the mean square error between the first single-step prediction result and the second single-step prediction result, and use it as the output consistency constraint term.

[0117] Using a mean squared error calculation model, the differences in pixel features at corresponding spatial locations between the first and second single-step prediction results are summed and averaged to output a scalar error value. This scalar error value is recorded by the system as an output consistency constraint term. By setting this constraint mechanism, the aim is to ensure that the spatial layout of the single-step output of the diffusion network converges as much as possible when processing the same compressed input, thereby reducing the bias effect of initial Gaussian random noise on the final generated pixel structure.

[0118] The calculation of the supervised reconstruction loss term and the construction of the joint loss function include steps 408 to 409: Step 408: Based on the single-step prediction results of the initial one-step diffusion generation network and the target reference data, calculate the supervised reconstruction loss term for generation quality.

[0119] After quantifying the noise immunity performance, the error between the network output feature tensor and the target benchmark is further extracted. The spatial distance and semantic difference between the reconstruction matrix generated in the current step and the corresponding target ground truth matrix are calculated, and the supervised reconstruction loss term is output. The specific evaluation logic used for this loss term and the source of the target ground truth extraction will be described in detail in the subsequent parallel implementation methods.

[0120] Step 409: Construct a joint loss function based on the output consistency constraint term and the supervised reconstruction loss term, and perform iterative updates of the network parameters.

[0121] After obtaining the two scalar errors with different optimization dimensions, the integrated gradient feedback target of the entire link is calculated using a weighted combination mechanism.

[0122] For example: loss = α * loss _dist +β*loss _cons ; In the formula, loss is the calculated value of the constructed joint loss function, α is the pre-configured supervised reconstruction weight coefficient, and loss _dist To calculate the extracted supervised reconstruction loss term, β is the pre-configured consistency constraint weight coefficient, and loss... _cons This is used to calculate the extracted output consistency constraints.

[0123] After the numerical calculation of the joint loss function is completed, the internal adaptive moment estimation optimization model, namely the Adam optimizer, is invoked. Based on the joint loss function value, the partial derivative matrix is ​​calculated in the perceptron topology, and then according to the preset 10... -5The network learning rate, on a massive scale, performs inverse step updates to network parameters such as convolutional kernels and attention weights at each layer. Hyperparameters involved in training, such as the learning rate, training steps, and moving average decay rate, can be adaptively adjusted using conventional hyperparameter tuning methods.

[0124] Step 410: Based on the training conditions and the random noise obtained from sampling, the initial latent variable fitting network and the initial one-step diffusion generation network are jointly fine-tuned end-to-end using a joint loss function to obtain the preset latent variable fitting network and one-step diffusion generation network.

[0125] Through multiple cascaded iterations of the aforementioned calculation and update mechanism, the initial latent variable adaptation network and the initial one-step diffusion generation network gradually adjust their feature mapping capabilities under the guidance of the gradient of the joint loss function. The training fine-tuning loop terminates when the number of training iterations reaches a preset numerical threshold (e.g., 50,000 steps) or the calculated value of the joint loss function converges to a dynamically set fluctuation range. The full weight state in the network model's memory register is then solidified and written to persistent storage, transforming it into the preset latent variable adaptation network and one-step diffusion generation network required for the online decoding and inference stage.

[0126] Example 5: As a preferred method for calculating the supervised reconstruction loss term in Example 4, this example proposes to use a knowledge distillation paradigm to guide a single-step network and calculate the loss in the latent space.

[0127] The training image samples are transformed into the latent space by the encoder corresponding to the VAE encoder or the inverse transform decoding module, so that the teacher model and the one-step network are in the same latent space; the multi-step teacher model outputs the denoised latent representation in the latent space; the reconstructed latent representation output by the one-step diffusion network and the denoised latent representation output by the teacher model (multi-step denoised pseudo-true value) are used to calculate the loss, and the two are spatially aligned and dimensionally aligned before calculating the error.

[0128] In one possible implementation, calculating the supervised reconstruction loss term for the generated quality includes the following steps: Step 501: Using a pre-built multi-step diffusion decoder as a teacher model, multi-step denoising is performed on the training image samples to generate multi-step denoised pseudo-real values.

[0129] Obtain the initial one-step diffusion generation network's single-step prediction results based on the training conditions.

[0130] During the knowledge distillation computation branch, pre-trained high-performance diffusion model weights are loaded into GPU memory. This pre-trained model is configured as a multi-step diffusion decoder and acts as a teacher model that passes diffusion prior knowledge in the current fine-tuning pipeline.

[0131] To ensure alignment of feature extraction logic, the teacher model and the single-step network to be fine-tuned share the same feature extraction backbone network structure. The teacher model receives training image samples as input from previous operations and, under its control, uses training conditions (compressed latent variables) as guidance. Starting from standard Gaussian noise, it initiates an implicit multi-step denoising inference flow within the latent space corresponding to the training image samples, outputting multi-step denoising pseudo-true values ​​in the same latent space as the single-step network output. For example, using the Denoising Diffusion Implicit Model Sampling (DDIM) algorithm, with a total of 50 iterations, multiple rounds of serial denoising feature inference calculations are performed on the input data.

[0132] After the aforementioned 50 iterative calculations, the teacher model generates a converged image feature matrix at the output, which is defined as the multi-step denoising pseudo-ground value. Compared to the original acquired physical image, the multi-step denoising pseudo-ground value possesses a smooth feature distribution after network mapping and contains rich high-dimensional generative network prior features. These prior features suppress random hard noise edges during the physical acquisition process, providing the subsequent student network with a soft-labeled supervision target that is numerically easier to fit.

[0133] Step 502: Calculate the approximation error between the single-step prediction result of the initial one-step diffusion generation network based on the training conditions and the pseudo-true value of the multi-step denoising, and use it as the supervised reconstruction loss term.

[0134] After obtaining the feature benchmark generated by the teacher model, the feature tensor output by the initial one-step diffusion generator network in the training state within a single forward propagation cycle is extracted simultaneously. This feature tensor is the single-step prediction result of the initial one-step diffusion generator network based on the training conditions.

[0135] Since the teacher model and the initial one-step diffusion generative network share the same backbone architecture and both infer within the same latent space, their outputs are naturally aligned in spatial resolution and channel dimension, requiring no additional projection or interpolation processing. In practical engineering implementations, if minor deviations in tensor shape occur due to batch dimension or edge clipping, those skilled in the art can achieve alignment through conventional tensor clipping or padding operations.

[0136] An error measurement operator is constructed to extract the difference between the single-step prediction result and the multi-step denoising pseudo-true value at corresponding pixel positions. The obtained residual feature matrix is ​​squared and averaged in terms of spatial dimensions to extract a scalar quantization value that measures the numerical distance between the two high-dimensional distributions.

[0137] The scalar quantization value is the approximation error. It is directly assigned and written into the storage medium, serving as the supervised reconstruction loss term required in Example 4 for subsequent joint gradient calculation. By setting this calculation operation, during the parameter fine-tuning stage, the feature distribution obtained from a single Gaussian random noise mapping is required to converge towards the endpoint of the denoised trajectory obtained after 50 iterations, thus achieving the transfer of image generation quality between network models with different computational time spans.

[0138] Furthermore, during the iterative update of network weights based on the supervised reconstruction loss term, an exponential moving average strategy can be introduced to smooth the network parameters. Specifically, an independent moving average copy of the weight matrix for each step of the network is maintained in memory.

[0139] After backpropagation is completed in each training step, the weight copy value of the current step is calculated by weighting and summing the smoothed weights from the previous historical step and the newly calculated gradient update weights according to a preset decay rate parameter. This preset decay rate parameter can be 0.9999. This time-series decay-based weighted update mechanism reduces the risk of parameter oscillations that may be caused by single-step high gradient backpropagation, maintaining the convergence stability of the fine-tuned network during the generation process.

[0140] According to one aspect of this application, θ ema (t) = λ × θ ema (t-1) + (1-λ) × θ(t), in θ ema θ(t) is the moving average weight at step t, θ(t) is the weight after gradient update at step t, and λ is the preset decay rate parameter.

[0141] Example 6: Unlike Embodiment 5 above, this embodiment provides an alternative calculation method for the supervised reconstruction loss term. Used in scenarios such as basic training or computationally limited training, this method improves training stability to some extent; high perceptual quality is preferably obtained further through multi-step teacher distillation or perceptual quality constraints.

[0142] The specific difference lies in the fact that it does not rely on a multi-step teacher model to generate pseudo-true values, but instead directly uses the original image data to participate in error extraction and calculates the loss in the image space. The main idea is as follows: First, perform an inverse transform decoding on the reconstructed latent representation output by the one-step diffusion network to obtain the training reconstructed image; the original image is an RGB image; calculate the supervised reconstruction loss by comparing the training reconstructed image with the original RGB image used as the ground truth.

[0143] The rest of the training architecture and fine-tuning mechanism are the same as in Example 4, and can be referred to the foregoing description, so they will not be repeated here.

[0144] In one possible implementation, the following steps are included: Step 601: Use the training image samples directly as the target ground truth.

[0145] When constructing and calculating the joint loss function, the inference loading and multi-step denoising derivation process of the teacher network are skipped. The tensor matrix corresponding to the training image samples in the original input pipeline is directly extracted through the data reading interface.

[0146] Depending on the specific computational space settings, this tensor matrix can be either the original red-green-blue color space matrix or a clean latent space representation matrix after being encoded and transformed by a variational autoencoder. In memory, this clean tensor matrix is ​​assigned to a specific target variable and defined as the target truth value.

[0147] This data scheduling strategy reduces the memory usage of the training cluster and the number of floating-point operations per parameter update cycle, making it suitable for model fine-tuning environments with limited computing power or missing basic pre-trained teacher models.

[0148] Step 602: Obtain the initial one-step prediction result of the one-step diffusion generator network based on the training conditions; calculate the reconstruction error between the initial one-step prediction result of the one-step diffusion generator network based on the training conditions and the target ground truth, and use it as the supervised reconstruction loss term.

[0149] After obtaining the target ground truth, it is spatially aligned with the single-step prediction result output by the initial one-step diffusion generation network within the current forward propagation cycle. A preset tensor distance calculation operator is invoked to operate on the two matrices, calculating their numerical residuals in each corresponding dimension.

[0150] The tensor distance calculation operator specifically employs either a mean squared error calculation model or an absolute value error calculation model. The calculated numerical residuals are then summed and averaged along the spatial dimension, transforming them into a scalar value reflecting the degree of deviation between the two sets of data distributions. This scalar value is the reconstruction error, which is extracted and recorded as a supervised reconstruction loss term, used for subsequent weighted merging with the output consistency constraint term.

[0151] Furthermore, in the fine-tuning process of a single-step diffusion network, directly using the target ground truth as a hard label for supervised calculation can easily lead to oscillations in the gradient descent trajectory, or cause the network to tend to output averaged smooth pixels in order to minimize statistical error.

[0152] To suppress this degradation, a joint regulation mechanism based on an introduced output consistency constraint is employed. In this configuration, the supervised reconstruction loss term provides macroscopic color and structure distribution anchors, while the output consistency constraint term maintains the robustness of the single-step feature extraction process to initial noise disturbances. The two mutually constrain each other, improving training stability to some extent while maintaining training efficiency. High perceptual quality still requires further optimization through multi-step teacher distillation or perceptual quality constraints.

[0153] In some embodiments, the single-step prediction result is decoded to the image space via inverse transformation, or the training image samples are transformed to the latent space via an encoder. After aligning the space and dimensions, the reconstruction error between the single-step prediction result and the target ground value is calculated as the supervised reconstruction loss term.

[0154] It should be noted that both of the above implementation methods require spatial alignment and dimensional alignment before error calculation.

[0155] Example 7: After the image decoding and generation are completed in the above embodiments, this embodiment provides an optional post-processing scheme to address the color drift phenomenon that is prone to occur in diffusion prior models.

[0156] In one possible implementation, the following steps are included: Step 701: Parse and obtain the color statistics of the original image from the image compression bitstream.

[0157] At the encoding end, the mean and standard deviation of each color channel of the original image are calculated, the mean and standard deviation are quantized and encoded, and written as supplementary metadata into the preset fields of the image compression bitstream.

[0158] When performing single-step diffusion generation, the fine-tuning correction step in the multi-step iterative denoising process is removed, which can easily cause the generated image tensor to deviate from the original true data distribution in terms of global color distribution.

[0159] In order to provide anchor data for color correction at the decoding end, the encoding end pre-calculates the statistical characteristics of the input image on each color channel when performing the front-end compression process, and quantizes the mean and standard deviation. Then, lossless entropy encoding is used to write it as supplementary metadata into the preset header field of the image compressed bitstream.

[0160] At the encoding end, the mean and standard deviation of each color channel of the original image are calculated, the mean and standard deviation are quantized and encoded, and written as supplementary metadata into a preset field of the image compressed bitstream; at the decoding end, color statistics are parsed from the preset field.

[0161] After completing the basic network inference, the decoder calls the bitstream reading operator to extract the header of the image compressed bitstream, separating the mean scalar and standard deviation scalar corresponding to each color channel. These extracted numerical sets constitute the color statistics of the original image. The above scalar data is transmitted along with the path with extremely low data overhead, providing a global color distribution reference for subsequent post-processing modules.

[0162] Step 702: Perform distribution renormalization on the reconstructed image based on color statistics to obtain the corrected reconstructed image.

[0163] At the decoding end, color statistics are parsed from preset fields, and the reconstructed image is subjected to distribution renormalization based on the color statistics to obtain the corrected reconstructed image.

[0164] After acquiring color statistics and the reconstructed image output by the inverse transform decoding module, independent computational memory is allocated in memory to execute pixel-based color correction logic. Based on the current pixel matrix of the reconstructed image, the actual pixel mean and actual pixel standard deviation in each feature channel are recalculated.

[0165] By introducing color statistics from the original image, distribution renormalization is performed on the coordinates of each pixel in the reconstructed image. The physical essence of this process is a first-order linear transformation under statistical constraints, used to force the alignment of the data distribution. The operational relationship is expressed as follows: I _corrected =( ( I _recon -M _recon ) / S _recon )*S _orig +M _orig ; Among them, I _corrected I represents the corresponding pixel channel value of the corrected reconstructed image. _recon To reconstruct the corresponding pixel channel values ​​of the image before transformation, M _recon S is the current color mean value extracted by the system for the reconstructed image. _recon S is the current color standard deviation extracted by the system for the reconstructed image. _orig M represents the source color standard deviation included in the color statistics of the original image. _orig The source color mean is included in the color statistics of the original image.

[0166] In engineering practice, degenerate scenarios such as all-zero images are generally not encountered, so the denominator is not 0. To be on the safe side, a small positive number can be added to avoid calculation errors.

[0167] By applying the above formula to each pixel, the color value distribution within the reconstructed image is translated and scaled. This operation makes the statistical moment characteristics of the output image more consistent with the actual data from the sending end. The target data matrix generated after this traversal calculation is confirmed as the corrected reconstructed image. This post-processing branch, as a compensation mechanism for an independent one-step inference network, suppresses the color cast problem in the generated image without increasing the inference load of a complex network.

[0168] In summary, the goal is to make the first- or second-order color statistics of the reconstructed image consistent with the corresponding statistics of the original image, thereby alleviating the overall color cast.

[0169] Example 8: A generative image compression and decoding system based on one-step diffusion is provided. This system can be used to execute the methods provided in the above embodiments, and its implementation principle and technical effects are similar. The same parts will not be repeated here. The following focuses on the module division, evaluation mechanism and application scenarios.

[0170] In one possible implementation, it includes: The decoding and parsing module is used to obtain the image compressed bitstream, perform entropy decoding and parsing on the image compressed bitstream, and obtain the compression latent variables; In physical hardware or basic software architecture, the decoding and parsing module is deployed within the network interface controller of the receiving terminal or the upper-layer application parsing protocol stack. This module captures the binary bitstream processed by the channel demodulation protocol, extracts discrete probability distribution information using built-in arithmetic decoding logic circuits or processor instruction sets, and reconstructs a three-dimensional floating-point tensor matrix based on this. The extracted compressed latent variables are written into the shared memory or cache of the terminal device, serving as the data source for subsequent computation modules.

[0171] The feature adaptation module is used to input compressed latent variables into a preset latent variable adaptation network for dimension mapping and feature space alignment, generating multi-scale conditional adaptation features. The feature adaptation module is implemented at the physical level by calling the matrix operation acceleration unit inside the digital signal processor. This acceleration unit loads the latent variable adaptation network weight data read from the read-only storage medium and performs channel transformation and spatial interpolation operations on the compressed latent variables originating from shared memory. Based on the bus bandwidth and clock frequency, this module calculates and outputs feature tensor data blocks of various resolution specifications, forming multi-scale conditional adaptation features, and stores them in an independent video memory area for the image processor to access.

[0172] The noise acquisition module is used to acquire initial Gaussian random noise of a preset dimension. The initial Gaussian random noise is a reproducible noise tensor, which is generated by a preset fixed seed, uses a preset noise tensor, or is determined by the bitstream field. The noise acquisition module utilizes a pseudo-random number generation interface based on a preset fixed seed, a preset noise tensor, or pseudo-random noise determined by the bitstream field to calculate reproducible noise data. This module receives request instructions from the generation network and constructs a corresponding tensor structure according to the four-dimensional shape parameters attached to the instructions. Subsequently, the module fills this tensor structure with floating-point values ​​conforming to a standard normal distribution, generating initial Gaussian random noise as the starting point for diffusion inference.

[0173] The single-step generation module is used to input multi-scale conditional adaptation features and initial Gaussian random noise into the one-step diffusion generation network. The corresponding scale conditional adaptation features are injected into the multiple scale feature levels of the one-step diffusion generation network and generated by a single forward propagation to obtain the reconstructed latent representation. The single-step generation module is the system's main inference computation unit, and its physical entity can be a graphics processing unit (GPU) equipped with a tensor core or a dedicated neural network computing chip. This computation unit synchronously reads multi-scale conditional adaptation features and initial Gaussian random noise from video memory, and utilizes the parallel computing capabilities of the GPU to perform dot product and weighted processing in cross-attention operations. Since the overall architecture is only designed to perform a single forward propagation of the computation graph, this module releases the currently occupied computational core resources after completing the computation and reconstructing the latent representation, thus controlling the overall power consumption of the inference process.

[0174] The inverse transform output module is used to perform inverse transform decoding on the reconstructed latent representation and output the reconstructed image.

[0175] The inverse transform output module contains computational logic for performing image space deconvolution and color space mapping. This module maps the reconstructed latent representation located in the high-dimensional feature space back to the red-green-blue color array and writes it to the frame buffer. During system testing or offline quality evaluation, a learned perceptual image patch similarity network model is typically used as an objective evaluation metric. Since this similarity network model requires the numerical distribution of the input data to be within a specific range of -1 to 1, a linear mapping calculation needs to be performed on the generated tensor using an associated test computation unit.

[0176] The corresponding normalized mapping formula is: img _norm =img*2-1; Among them, img _norm The target evaluation tensor is normalized, and `img` is the original image tensor generated by the inverse transform output module, with values ​​limited to the range of 0 to 1. The target evaluation tensor obtained after normalization is input into the perceptual evaluation network to calculate the error scalar. This calculation process does not involve gradient back-update of system parameters and is used to extract evaluation values ​​to perform performance comparisons between systems.

[0177] In a series of experimental tests aimed at verifying the effectiveness of the method of the present invention, the method of the present invention was compared with the traditional multi-step diffusion decoding method using the Kodak public test image set. Under the same compression bit rate, the method of the present invention only requires a single forward propagation to complete image reconstruction, and the decoding time is reduced by more than an order of magnitude compared with the 50-step iteration multi-step diffusion method.

[0178] On the perceptual quality evaluation metric LPIPS, the perceptual quality difference between the method of this invention and the multi-step diffusion decoding method remains within an acceptable range, while performing comparablely on the Fraser initiation distance (FID) distribution distance metric. Those skilled in the art will understand that specific performance metric values ​​may vary depending on the test dataset, compression bitrate settings, and hardware platform configuration.

[0179] The generative image compression and decoding method based on one-step diffusion can be implemented in low-bandwidth image communication systems, mobile terminal image sharing applications, or cloud image rendering pipelines.

[0180] The acquisition of the image compressed bitstream specifically involves receiving the image compressed bitstream via a low-bandwidth wireless channel; the inverse transform decoding of the reconstructed latent representation is performed on the neural network inference acceleration chip of the mobile terminal.

[0181] When this method is compiled and deployed in a low-bandwidth image communication system, it exists as an image reconstruction algorithm component of the underlying communication protocol for video conferencing. Leveraging its low bandwidth requirements and short decoding computation time, the system recovers the visual frame sequence with low decoding latency under network congestion conditions, thus controlling image transmission latency in channel-constrained scenarios.

[0182] When applied to mobile terminal image sharing applications, this algorithm is integrated into the local application logic of the mobile device. When the device detects that it is in a low signal coverage threshold area, it triggers the compression and decoding pipeline to reduce the number of bytes required for local image data to be transmitted uplink.

[0183] When deployed in a cloud-based image rendering pipeline, this method performs source-side encoding and compression on high-resolution scene frames rendered on the cloud desktop server. The user terminal receives the low-bitrate data and then reconstructs the scene image through a single forward diffusion inference. This approach reduces the bandwidth threshold required for establishing data transmission links in cloud rendering services.

Claims

1. A generative image compression and decoding method based on one-step diffusion, characterized in that, include: Obtain the compressed image bitstream, perform entropy decoding and parsing on the compressed image bitstream to obtain the compression latent variables; The compressed latent variables are input into a pre-defined latent variable adaptation network for dimension mapping and feature space alignment to generate multi-scale conditional adaptation features. Obtain initial Gaussian random noise of a preset dimension. The initial Gaussian random noise is a reproducible noise tensor, generated by a preset fixed seed, or using a preset noise tensor, or determined by the bitstream field. The conditional adaptation features at multiple scales and the initial Gaussian random noise are input into the one-step diffusion generation network. The corresponding scale conditional adaptation features are injected into the multiple scale feature levels of the one-step diffusion generation network and generated by a single forward propagation to obtain the reconstructed latent representation. The reconstructed latent representation is decoded by inverse transform, and the reconstructed image is output.

2. The method according to claim 1, characterized in that, The compressed latent variables are input into a pre-defined latent variable adaptation network for dimensionality mapping and feature space alignment, generating multi-scale conditional adaptation features, including: The compressed latent variables are reduced in dimensionality by using a pre-constructed first convolutional layer to obtain dimensionality-reduced features. Spatial mapping features of the dimensionality reduction features are extracted using a pre-constructed second convolutional layer to obtain spatial modulation features; By using a pre-constructed third convolutional layer, the number of channels of the spatial modulation features is mapped to a preset target dimension to obtain the basic adaptation features; Multi-scale resolution interpolation is performed on the basic adaptation features to generate multi-scale conditional adaptation features.

3. The method according to claim 1, characterized in that, The one-step diffusion generation network has an encoder-decoder architecture and multiple scale feature levels, including sequentially connected downsampling stage feature maps, bottleneck layer feature maps, and upsampling stage feature maps. Conditional adaptation features of corresponding scales are injected into multiple scale feature layers of the one-step diffusion generation network, including: The conditional adaptation features of the corresponding scale are injected into the feature maps of the downsampling stage and the upsampling stage respectively to construct generation constraints for local texture and high-frequency detail information. Conditional adaptation features of the corresponding scale are injected into the bottleneck layer feature map to construct generation constraints for global semantic and structural consistency.

4. The method according to claim 1, characterized in that, Conditional adaptation features of corresponding scales are injected into multiple scale feature layers of the one-step diffusion generation network, including: By using a cross-attention mechanism, the conditional adaptation features of the corresponding scale are injected into the current generated features at the corresponding scale level, thereby realizing information interaction between the conditional adaptation features and the current generated features.

5. The method according to claim 4, characterized in that, By employing a cross-attention mechanism, conditionally adapted features at the corresponding scale are injected into the currently generated features at the corresponding scale level, including: Map the currently generated features to the query vector; The conditional adaptation features at the corresponding scales are mapped to key vectors and value vectors, respectively. Based on the similarity matching results between the query vector and the key vector, the value vector is weighted and summed to obtain the feature modulation result; Update the currently generated features using the feature modulation results.

6. The method according to claim 1, characterized in that, During the process of injecting conditional adaptation features of corresponding scales into multiple scale feature layers of the one-step diffusion generation network and generating them in a single forward propagation, no probability-based zeroing operation is performed on the conditional adaptation features to maintain the data integrity of the conditional adaptation features that serve as the generation guide condition.

7. The method according to claim 1, characterized in that, Before acquiring the image compressed bitstream, performing entropy decoding and parsing on the image compressed bitstream to obtain the compressed latent variables, there is also a model pre-construction stage, which includes: Obtain training image samples; The compressed latent variables corresponding to the training image samples are extracted based on the preset image coding components and used as training conditions. The initial latent variable fitting network and the initial one-step diffusion generation network are fine-tuned end-to-end using a joint loss function to obtain the preset latent variable fitting network and one-step diffusion generation network.

8. The method according to claim 7, characterized in that, The initial latent variable fitting network and the initial one-step diffusion generation network are jointly fine-tuned end-to-end using a joint loss function, including: Calculate the output consistency constraint term for different initial noise perturbations; Calculate the supervised reconstruction loss term for the generated quality; A joint loss function is constructed based on the output consistency constraint term and the supervised reconstruction loss term, and the network parameters are iteratively updated.

9. A generative image compression and decoding system based on one-step diffusion, characterized in that, include: The decoding and parsing module is used to obtain the image compressed bitstream, perform entropy decoding and parsing on the image compressed bitstream, and obtain the compression latent variables; The feature adaptation module is used to input compressed latent variables into a preset latent variable adaptation network for dimension mapping and feature space alignment, generating multi-scale conditional adaptation features. The noise acquisition module is used to acquire initial Gaussian random noise of a preset dimension. The initial Gaussian random noise is a reproducible noise tensor, which is generated by a preset fixed seed, or by using a preset noise tensor, or determined by the bitstream field. The single-step generation module is used to input multi-scale conditional adaptation features and initial Gaussian random noise into the one-step diffusion generation network. The corresponding scale conditional adaptation features are injected into the multiple scale feature levels of the one-step diffusion generation network and generated by a single forward propagation to obtain the reconstructed latent representation. The inverse transform output module is used to perform inverse transform decoding on the reconstructed latent representation and output the reconstructed image.

10. A model construction method for one-step diffusion generative image compression and decoding, characterized in that, include: Obtain training image samples; Based on a preset image coding component, the compressed latent variables corresponding to the training image samples are extracted, and the compressed latent variables are used as training conditions. The training conditions are input into the initial latent variable adaptation network for dimension mapping and feature space alignment to generate multi-scale training condition adaptation features. Based on the multi-scale training condition adaptation features and the initial Gaussian random noise, the initial one-step diffusion generation network is made to perform a single forward propagation to obtain a single-step prediction result. Calculate the output consistency constraint term for different initial noise perturbations; Calculate the supervised reconstruction loss term for the generated quality; A joint loss function is constructed based on the output consistency constraint term and the supervised reconstruction loss term; Based on the joint loss function, the initial latent variable adaptation network and the initial one-step diffusion generation network are jointly fine-tuned end-to-end to obtain the latent variable adaptation network and the one-step diffusion generation network for image compression and decoding.

Images