Learning type image compression method based on stream matching residual reconstruction

Abstract

The invention provides a learning type image compression method based on stream matching residual reconstruction, and relates to the technical field of image processing and computer vision. The method comprises the following steps: constructing an image compression network model based on a flow matching latent variable refining module; wherein the flow matching latent variable refining module performs residual error reconstruction on quantitative latent variables by using a time continuous microfluidical field, and compensates for fine details lost in a quantization process; constructing a joint optimization objective function based on a rate distortion loss function and a stream matching loss function, and performing end-to-end training on the whole image compression network model by using an input image; and performing image compression on a to-be-compressed image by using the trained image compression network model. By adopting the method, fine details lost in the quantization process can be effectively compensated under the condition that reasoning delay is not remarkably increased, and the structural fidelity and perception quality of image reconstruction are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a learning-based image compression method based on flow matching residual reconstruction. Background Technology

[0002] Image compression is one of the important foundational technologies in the field of information processing. Its goal is to maintain high reconstruction quality while minimizing storage and transmission overhead.

[0003] Traditional compression algorithms (such as JPEG, H.265, and H.266) are mainly based on manually designed discrete transform and quantization schemes. Although these methods are simple to implement, their fixed block structure is prone to introducing obvious block artifacts and texture distortion at low bit rates, making it difficult to meet the compression requirements of modern high-resolution, multi-scene images.

[0004] In recent years, the development of deep learning technology has given rise to Learned Image Compression (LIC) methods. These methods replace traditional hand-designed modules with end-to-end trained neural networks, surpassing classic codecs in many performance metrics. Among them, compression models based on Variational Autoencoders (VAEs) can effectively optimize the rate-distortion balance; while compression methods based on generative models (such as GANs, Diffusion, and Flow Matching) have advantages in perceptual quality.

[0005] However, current learning-based compression frameworks still suffer from the following technical bottlenecks: 1. Quantization residuals are difficult to recover accurately: Quantization introduces information loss and irreversible noise, which limits the expressive power of latent variables; 2. There is a trade-off between perceived quality and structural distortion: VAE models tend to prioritize structural fidelity, while generative models tend to prioritize visual realism. Therefore, there is an urgent need for a new, efficient generative compression framework that can effectively reconstruct quantized residual information while maintaining low computational costs, and achieve a better balance between rate distortion and perceived metrics. Summary of the Invention

[0006] To address the technical problems of inaccurate recovery of quantization residuals and the trade-off between perception quality and structural distortion in existing technologies, this invention provides a learning-based image compression method based on flow-matching residual reconstruction. The technical solution is as follows:

[0007] On the one hand, a learning-based image compression method based on flow matching residual reconstruction is provided. This method is implemented by a learning-based image compression device and includes: An image compression network model based on a flow-matching latent variable refinement module is constructed. The flow-matching latent variable refinement module uses a time-continuously differentiable flow field to perform residual reconstruction on the quantized latent variables, compensating for the fine details lost during the quantization process. A joint optimization objective function based on rate-distortion loss function and flow matching loss function is constructed, and the entire image compression network model is trained end-to-end using the input image; The trained image compression network model is used to compress the image to be compressed.

[0008] Furthermore, the image compression network model includes: an analysis transform network. Hyperanalysis Transformation Network Hypersynthetic Transformation Network Stream matching latent variable refinement module and synthesis transformation network ; The processing flow of the image compression network model includes: By analyzing the transform network Input image Encoded as the first latent representation ; Through hyperanalysis transformation network The first potential representation Encoding as a second latent representation and to and Quantification is performed to obtain the latent variables. and ; Based on quantified latent variables Through hypersynthetic transformation network Generate entropy model parameters, where the entropy model parameters include the mean. and standard deviation mean Used to implement Quantization operations; The latent variable refinement module, using a time-continuously differentiable flow field, quantifies latent variables. Modeling to predict quantized residuals The predicted quantization residual With quantification of latent variables Fusion yields corrected latent variables ; Through synthetic transformation network For the modification of latent variables Decode the image to generate a reconstructed image. .

[0009] Furthermore, regarding Quantification is performed to obtain the latent variables. : ; In the formula, For quantization operations, μ is a value derived from the supersynthesis transformation network. Output mean parameter .

[0010] Furthermore, the flow matching latent variable refinement module uses time variables... For a continuous path, through the velocity field function Construct ordinary differential equations: ; In the formula, It is a mapping function that changes over time; This is the initial condition of the mapping function, that is, in At that time, the mapping function is an identity function, and its value is the quantized latent variable. ; The ordinary differential equation is used to gradually transform the fundamental distribution into the target residual distribution in order to predict the quantized residual. .

[0011] Furthermore, the latent variables were modified. Represented as: ; .

[0012] Furthermore, the stream matching latent variable refinement module includes: a time embedding module and N stacked residual blocks; wherein, The time embedding module is used to receive and process a time step information; Stacked residual blocks are used to quantize latent variables. The time parameter table output by the time embedding module is processed to obtain the corrected latent variable. .

[0013] Furthermore, each residual block includes: The main path includes: a first Conv-LN module and a second Conv-LN module; each Conv-LN module includes, in sequence: 3 The first Conv-LN module is connected to the second Conv-LN module sequentially through scaling and shifting operations and residual connections, consisting of three convolutional layers and a layer normalization layer. The time adjustment path is used to receive the time parameter table output from the time embedding module and generate scaling parameters accordingly. and displacement parameters ; A scaling displacement operation is used to perform an affine transformation on the output of the first Conv-LN module in the main path using the α and β; The residual connection adds the input of the residual block to the output of the scaling displacement operation.

[0014] Furthermore, the time adjustment path, used to generate α and β, comprises, in sequence: a linear layer, a layer normalization layer, and a 1... 1. Convolutional layer.

[0015] Furthermore, the constructed joint optimization objective function Represented as: ; In the formula, For bit rate loss, To reconstruct the distortion, For stream matching loss, For Lagrange multipliers, These are hyperparameters used to balance the two types of losses. For the input image, To reconstruct the image.

[0016] Furthermore, the flow matching loss Used to minimize the predicted velocity field Reference velocity field The difference between them is in the form of: ; In the formula, It refers to time Find the expected value. Represented as L2 norm, It is the velocity field learned by the flow matching latent variable refinement module. It is a known reference vector field. From the basic distribution To data distribution The intermediate probability density along the path.

[0017] On the other hand, a learning-based image compression device is provided, the learning-based image compression device comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement any of the above-described learning-based image compression methods based on stream matching residual reconstruction.

[0018] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described learning-based image compression methods based on stream matching residual reconstruction.

[0019] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: 1) The stream matching latent variable refinement module can effectively compensate for the fine details lost during the quantization process without significantly increasing inference latency.

[0020] 2) Construct a joint optimization objective function based on rate-distortion loss function and flow matching loss function, and use the input image to train the entire image compression network model end-to-end, which significantly improves the structural fidelity and perceptual quality of image reconstruction.

[0021] 3) It is both versatile and pluggable, and can be adapted to a variety of learning-based image compression frameworks. It can also be widely used in image encoding, video transmission, edge computing and artificial intelligence vision systems. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a learning-based image compression method based on flow matching residual reconstruction provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a learning-based image compression network based on flow matching residual reconstruction provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the Residual flowmatching for Learned Image Compression (ReLIC) module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the rate-distortion (RD) performance curves of the learning-based image compression method based on flow matching residual reconstruction and the baseline method provided in this embodiment of the invention on the Kodak dataset; Figure 5 This is a schematic diagram of the rate-distortion (RD) performance curves of the learning-based image compression method based on flow matching residual reconstruction and the baseline method provided in this embodiment of the invention on the CLIC.m dataset; Figure 6 This is a schematic diagram of the rate-distortion (RD) performance curves of the learning-based image compression method based on flow matching residual reconstruction and the baseline method provided in this embodiment of the invention on the CLIC.p dataset; Figure 7This is a schematic diagram of the structure of a learning-type image compression device provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0025] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0026] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0027] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0028] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0029] This invention provides a learning-based image compression method based on flow matching residual reconstruction. This method can be implemented using a learning-based image compression device, which can be a terminal or a server. Figure 1 The flowchart shown is for a learning-based image compression method based on flow matching residual reconstruction. The processing flow of this method may include the following steps:

[0030] S1. Construct an image compression network model based on the flow matching latent variable refinement module; wherein, the flow matching latent variable refinement module uses a time-continuously differentiable flow field to perform residual reconstruction on the quantized latent variables, compensating for the fine details lost during the quantization process; The classic learning-based image compression framework includes: analyzing the transform network. (Encoder), used to process input images Extraction as a latent representation Hyperanalysis Transformation Network (Encoder), used to extract bypass information Hypersynthetic Transformation Network (Decoder), used to convert the quantized data Convert to Gaussian distribution parameters Synthetic Transformation Network (Decoder), used to reconstruct images .

[0031] In the classic LIC framework, after the analytical transform network maps the input image to the latent space, the quantization operation inevitably introduces distortion, causing the latent variables to lack the ability to express high frequencies and fine structures. This invention's embodiment maintains the original analytical transform network... Hyperanalysis Transformation Network Hypersynthetic Transformation Network With synthetic transformation network Without changing the overall architecture, a new module called ReLIC, specifically designed for refining latent variables in stream matching to handle quantization loss, is added. Figure 2 As shown, this module directly acts on the quantized latent variables and is the only component in the entire system used to actively recover the information lost in quantization. Therefore, it occupies a core position in the entire image compression network model provided in this embodiment of the invention.

[0032] like Figure 2 As shown, the image compression network model further includes an arithmetic encoder and an arithmetic decoder; the arithmetic encoder and arithmetic decoder are used to encode the quantized latent variables into codewords for transmission.

[0033] In this embodiment, the processing flow of the image compression network model includes: S11, By analyzing the transformation network Input image Encoded as the first latent representation ; In this embodiment, the transformation network is analyzed. A convolutional neural network, consisting of multiple convolutional layers and normalization layers with non-linear activation functions, is used to encode the input image layer by layer into low-dimensional latent spatial features. .

[0034] S12, Transform the network via hyperanalysis The first potential representation Encoding as a second latent representation and to and Quantification is performed to obtain the latent variables. and ; In this embodiment, the hyperanalysis transform network Used to extract the first latent representation It extracts contextual information and outputs the super-prior features, i.e., the second latent representation. .

[0035] In this embodiment, for Quantification is performed to obtain the latent variables. :

[0036] In the formula, For quantization operations, μ is a value derived from the supersynthesis transformation network. Output mean parameter .

[0037] In the embodiment, for Quantification is performed to obtain the latent variables. :

[0038] In the formula, For quantification operations.

[0039] S13. Based on quantified latent variables Through hypersynthetic transformation network Generate entropy model parameters, where the entropy model parameters include the mean. and standard deviation mean Used to implement Quantization operations; In this embodiment, the prior features After quantization, the input is given to the hypersynthetic transform network. To generate quantitative latent variables Gaussian distribution parameters ,Establish Probability distribution, thus enabling adaptive entropy modeling.

[0040] S14. Using the flow matching latent variable refinement module, the latent variables are quantified through a time-continuously differentiable flow field. Modeling to predict quantized residuals The predicted quantization residual With quantification of latent variables Fusion yields corrected latent variables ; like Figure 3 As shown, the stream matching latent variable refinement module includes: a time embedding module and N stacked residual blocks, wherein the intermediate residual blocks output the progressively refined latent variables, and the last residual block outputs the final refined latent variables. ;in, The time embedding module is used to receive and process a timestep information; Stacked residual blocks are used to quantize latent variables. (Dimensions are 16) 16 192) and the time parameter table output by the time embedding module are processed to obtain the corrected latent variable. .

[0041] In this embodiment, each residual block includes: The main path includes: a first Conv-LN (convolutional-layer normalization) module and a second Conv-LN module; each Conv-LN module includes: 3 The system consists of three convolutional layers and layer normalization (Layer Norm) layers. The first Conv-LN module is connected to the second Conv-LN module sequentially through scaling and shifting operations and residual connections. The time adjustment path is used to receive the time parameter table output from the time embedding module and generate scaling parameters accordingly. and displacement parameters ; A scale shift operation is used to perform an affine transformation on the output of the first Conv-LN module in the main path using the α and β. The residual connection adds the input of the residual block to the output of the scaling displacement operation.

[0042] In this embodiment, the time adjustment path is used to generate α and β, and its structure includes, in sequence: a linear layer, a layer normalization layer, and a 1 1. Convolutional layer.

[0043] In this embodiment, the flow-matching latent variable refinement module uses flow-matching theory to construct a residual prediction network for residual reconstruction. Its core idea is to establish a continuously differentiable flow field from a simple basic distribution to a complex quantized residual distribution, enabling the model to progressively construct the reconstruction path of latent variables over time. The specific process is as follows:

[0044] 1. Input: Quantified latent variables With random sampling time variables ; 2. Time Embedding: embedding time variables Mapped to a high-dimensional representation, and conditional adjustments performed in each residual block; 3. Flow field construction: through velocity field network Estimate in time variable The forward flow direction; In this embodiment, the flow matching latent variable refinement module uses time variables. For a continuous path, through the velocity field function Construct ordinary differential equations: ; In the formula, It is a mapping function that changes over time; This is the initial condition of the mapping function, that is, in When the mapping function is an identity function, it takes the value of , indicates a sample taken from the data distribution, specifically referring to the quantified latent variables. .

[0045] In this embodiment, the ordinary differential equation is used to gradually transform the basic distribution into the target residual distribution in order to predict the quantized residual. .

[0046] 4. Residual prediction: Based on flow matching theory, the velocity field is integrated to obtain an estimate of the quantized residual; ; 5. Refine latent variables, that is: modify latent variables Represented as: ; In this embodiment, the formula can also be used: Indicates; among which, ReLIC For stream matching latent variable refinement module.

[0047] Unlike traditional models that rely on the decoder to fill in the details, this embodiment restores the latent variables to a more closely unquantized continuous representation by explicitly reconstructing the quantized residuals.

[0048] In this embodiment, the latent variables are corrected. Represented as:

[0049]

[0050] In the formula, ReLIC For stream matching latent variable refinement module.

[0051] In this embodiment, the number of sampling time steps is set to 5 to achieve a balance between compression performance (such as evaluation loss) and decoding latency; wherein, when the number of sampling time steps is less than 3, the generation quality will be reduced, while when it is greater than 25, the inference latency will increase significantly.

[0052] S15, Through the synthetic transformation network For the modification of latent variables Decode the image to generate a reconstructed image. .

[0053] In this embodiment, the synthetic transformation network To create a decoding network symmetrical to the analysis transform network structure, it includes multiple deconvolutional layers, upsampling layers, and nonlinear activation functions to correct latent variables. Restore to the output image .

[0054] S2. Construct a joint optimization objective function based on rate distortion loss function and flow matching loss function, and use the input image to train the entire image compression network model end-to-end; In this embodiment, the basic rate-distortion loss Used to optimize the compression ratio (Rate) of image compression networks. ) and distortion Reconstruction distortion Mean squared error (MSE) is typically used. The total rate distortion loss is... ,in, For bit rate loss, It is a Lagrange multiplier. For the input image, To reconstruct the image.

[0055] In addition, this embodiment introduces a flow matching loss. To specifically train the ReLIC module for minimizing the predicted velocity field With reference velocity field The difference between them is in the form of: ; in, It refers to time Find the expected value. Represented as L2 norm, It is the velocity field learned by the ReLIC module. It is a known reference vector field. From the basic distribution To data distribution The intermediate probability density along the path.

[0056] Joint optimization objective of the entire image compression network It is the weighted sum of the two: ; In the formula, It is a hyperparameter used to balance the two types of losses.

[0057] In this embodiment, the hyperparameter ω ranges from 0.01 to 0.1, which is used to make the overall behavior of the method closer to the variational autoencoder (VAE) baseline model, that is, to focus on optimizing the distortion metric while using the ReLIC module to improve the perceptual quality. When ω=0.05, it is possible to obtain the best perceptual quality of learned perceptual image patch similarity (LPIPS) while maintaining a high peak signal-to-noise ratio (PSNR).

[0058] In this embodiment, the first 300,000 images from the OpenImage dataset are used for training. During training, the input images are randomly cropped. Image patches of a certain size, with a batch size of 16. The model is trained using the Adam optimizer, with a learning rate of [value missing] for the first 36 epochs. The learning rate for the last four cycles was set to .

[0059] S3. Use the trained image compression network model to compress the image to be compressed.

[0060] In summary, the learning-based image compression method based on flow matching residual reconstruction provided in this invention has at least the following beneficial effects: 1) The stream matching latent variable refinement module can effectively compensate for the fine details lost during the quantization process without significantly increasing inference latency.

[0061] 2) Construct a joint optimization objective function based on rate-distortion loss function and flow matching loss function, and use the input image to train the entire image compression network model end-to-end, which significantly improves the structural fidelity and perceptual quality of image reconstruction.

[0062] 3) It is both versatile and pluggable, and can be adapted to a variety of learning-based image compression frameworks. It can also be widely used in image encoding, video transmission, edge computing and artificial intelligence vision systems.

[0063] To verify the effectiveness of the learning-based image compression method based on flow matching residual reconstruction provided in this embodiment of the invention, three standard test sets were used: Kodak, CLIC professional (CLIC.p), and CLIC mobile (CLIC.m).

[0064] To verify the plug-and-play nature of the ReLIC module of this invention, this embodiment selects two different baseline models for comparison: Mbt-mean (without LRP) and ELIC (with LRP). LRP (Learned Residual Prediction) is a residual compensation module commonly used in existing image compression paradigms.

[0065] Evaluation metrics: Pixel-level distortion measures PSNR and MS-SSIM, and reference-aware measures LPIPS and CLIP similarity.

[0066] The experimental results for rate-distortion performance (RD performance) are shown in Tables 1-3 and Figures 4-6 As shown.

[0067] Table 1 shows the performance comparison of the proposed method on the Kodak dataset.

[0068] Table 2 compares the performance of the proposed method on the CLIC.m dataset.

[0069] Table 3 shows the performance comparison of the proposed method on the CLIC.p dataset.

[0070] For the baseline ELIC using LRP, the method of this invention (ELIC + ReLIC) achieves BD-Rate improvements of -2.49%, -3.23%, and -2.50% (bitrate savings) on the Kodak, CLIC.m, and CLIC.p datasets, respectively.

[0071] For the baseline Mbt-mean without LRP, the method of this invention (Mbt-mean + ReLIC) achieves a higher BD-Rate improvement of -7.84%, -9.43%, and -9.88% on the three datasets, respectively.

[0072] Furthermore, the method of this invention also achieved a significant reduction in the perception metric (BD-LPIPS) (the lower the better). This demonstrates that ReLIC can improve models without LRP, and can also be combined with LRP to achieve superior results.

[0073] Efficiency Analysis: As shown in Table 4, since the ReLIC module primarily operates at the decoding end, the method of this invention does not introduce any additional latency at the encoding end (Enc.). At the decoding end (Dec.), the additional latency introduced by ReLIC (e.g., Mbt-mean increases from 66.8ms to 70.3ms) is moderate relative to its RD performance improvement. Compared to the diffusion model, stream matching offers a significant speed advantage.

[0074] Table 4. Comparison of complexity between the proposed method and the baseline method

[0075] Ablation Study: Sampling Time Step: The number of sampling steps affects generation quality and inference speed. As shown in Table 5, the latency is lowest when the number of sampling steps is 1 (200.9 ms), but a better evaluation loss (0.624) is obtained when the number of steps is 5. To balance performance and encoding time, this embodiment selects a sampling time step of 5.

[0076] Table 5 Ablation experiments with different sampling steps

[0077] Hyperparameters This parameter balances the VAE baseline (distortion) and the generative model (perception). As shown in Table 6, The larger the value, the closer the model is to the generative model (lower bitrate and PSNR, but better perception). To ensure the VAE baseline dominates, this embodiment selects... .

[0078] Table 6 Different hyperparameters ablation experiment

[0079] Figure 7 This is a schematic diagram of the structure of a learning-type image compression device provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the learning-type image compression device 710 may include a first processor 2001.

[0080] Optionally, the learning image compression device 710 may also include a memory 2002 and a transceiver 2003.

[0081] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0082] The following is combined with Figure 7 A detailed description of each component of the learning-type image compression device 710 is provided below: The first processor 2001 is the control center of the learning-type image compression device 710. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0083] Optionally, the first processor 2001 can perform various functions of the learning image compression device 710 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0084] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 are shown in the diagram.

[0085] In a specific implementation, as one example, the learning-type image compression device 710 may also include multiple processors, for example... Figure 7 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0086] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0087] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the learning image compression device 710. Figure 7 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0088] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0089] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 7 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0090] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the learning image compression device 710. Figure 7 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0091] It should be noted that, Figure 7 The structure of the learning image compression device 710 shown does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0092] Furthermore, the technical effect of the learning-type image compression device 710 can be referred to the technical effect of the learning-type image compression method based on flow matching residual reconstruction described in the above method embodiments, and will not be repeated here.

[0093] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0094] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0095] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0096] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0097] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0098] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0099] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0100] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0101] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0104] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A learning-based image compression method based on flow matching residual reconstruction, characterized in that, The method includes: An image compression network model based on a flow-matching latent variable refinement module is constructed. The flow-matching latent variable refinement module uses a time-continuously differentiable flow field to perform residual reconstruction on the quantized latent variables, compensating for the fine details lost during the quantization process. A joint optimization objective function based on rate-distortion loss function and flow matching loss function is constructed, and the entire image compression network model is trained end-to-end using the input image; The trained image compression network model is used to compress the image to be compressed.

2. The learning-based image compression method based on flow matching residual reconstruction according to claim 1, characterized in that, The image compression network model includes: an analysis transform network. Hyperanalysis Transformation Network Hypersynthetic Transformation Network Flow matching latent variable refinement module and synthesis transformation network ; The processing flow of the image compression network model includes: By analyzing the transform network Input image Encoded as the first latent representation ; Through hyperanalysis transformation network The first potential representation Encoding as a second latent representation and to and Quantification is performed to obtain the latent variables. and ; Based on quantified latent variables Through hypersynthetic transformation network Generate entropy model parameters, where the entropy model parameters include the mean. and standard deviation mean Used to implement Quantization operations; The latent variable refinement module, using a time-continuously differentiable flow field, quantifies latent variables. Modeling to predict quantized residuals The predicted quantization residual With quantification of latent variables Fusion yields corrected latent variables ; Through the synthetic transformation network For the modification of latent variables Decode and generate a reconstructed image. .

3. The learning-based image compression method based on flow matching residual reconstruction according to claim 2, characterized in that, right Quantification is performed to obtain the latent variables. : ； In the formula, For quantization operations, μ is a value derived from the supersynthesis transformation network. Output mean parameter .

4. The learning-based image compression method based on flow matching residual reconstruction according to claim 1, characterized in that, The flow matching latent variable refinement module uses time variables. For a continuous path, through the velocity field function Construct ordinary differential equations: ； In the formula, It is a mapping function that changes over time; This is the initial condition of the mapping function, that is, in At that time, the mapping function is an identity function, and its value is the quantized latent variable. ; The ordinary differential equation is used to gradually transform the basic distribution into the target residual distribution in order to predict the quantized residual. .

5. The learning-based image compression method based on flow matching residual reconstruction according to claim 4, characterized in that, Correcting latent variables Represented as: ； 。 6. The learning-based image compression method based on flow matching residual reconstruction according to claim 1, characterized in that, The stream matching latent variable refinement module includes: a time embedding module and N stacked residual blocks; wherein, The time embedding module is used to receive and process a time step information; Stacked residual blocks are used to quantize latent variables. The time parameter table output by the time embedding module is processed to obtain the corrected latent variable. .

7. The learning-based image compression method based on flow matching residual reconstruction according to claim 6, characterized in that, Each residual block includes: The main path includes: a first Conv-LN module and a second Conv-LN module; each Conv-LN module includes, in sequence: 3 The first Conv-LN module is connected to the second Conv-LN module sequentially through scaling and shifting operations and residual connections, consisting of three convolutional layers and a layer normalization layer. The time adjustment path is used to receive the time parameter table output from the time embedding module and generate scaling parameters accordingly. and displacement parameters ; A scaling displacement operation is used to perform an affine transformation on the output of the first Conv-LN module in the main path using the α and β; The residual connection adds the input of the residual block to the output of the scaling displacement operation.

8. The learning-based image compression method based on flow matching residual reconstruction according to claim 7, characterized in that, The time adjustment path, used to generate α and β, comprises, in sequence: a linear layer, a layer normalization layer, and a 1...

1. Convolutional layer.

9. The learning-based image compression method based on flow matching residual reconstruction according to claim 1, characterized in that, Constructed joint optimization objective function Represented as: ； In the formula, For bit rate loss, To reconstruct the distortion, For stream matching loss, For Lagrange multipliers, These are hyperparameters used to balance the two types of losses. For the input image, To reconstruct the image.

10. The learning-based image compression method based on flow matching residual reconstruction according to claim 9, characterized in that, The flow matching loss Used to minimize the predicted velocity field Reference velocity field The difference between them is in the form of: ； In the formula, It refers to time Find the expected value. Represented as L2 norm, It is the velocity field learned by the flow matching latent variable refinement module. It is a known reference vector field. From the basic distribution To data distribution The intermediate probability density along the path.

Images