Image reconstruction method, device and medium based on potential diffusion model

Abstract

The present application discloses an image reconstruction method, device, and medium based on a latent diffusion model, which relates to the field of image data processing. The method includes: using an encoder network to obtain mean and standard deviation tensors, generating latent variables based on these two tensors and random noise; using a cosine scheduling strategy to dynamically control the noise addition intensity and forward diffuse the latent variables; inputting the damaged image into a conditional encoder to obtain conditional information, and then performing a high-dimensional space projection operation to obtain a conditional embedding; using a cross-attention mechanism to conditionally enhance the latent variables after forward diffusion based on the conditional embedding; inputting the time step encoding, the conditionally enhanced latent variables, and the conditional embedding into a noise prediction network to obtain predicted noise, and then using a denoising diffusion implicit model for reverse diffusion to obtain a reconstructed image. The present application can reduce computational overhead, improve image reconstruction efficiency, and retain important details of the original image, thereby improving the quality of image reconstruction.

Description

Technical Field

[0001] The present application relates to the field of image data processing, and in particular to an image reconstruction method, device, and medium based on a potential diffusion model. Background Art

[0002] In the surveillance field, surveillance images and videos often suffer from blurry images due to factors such as weather, lighting, and aging hardware. This is particularly common in foggy weather, low-light conditions, or complex, dynamic scenes. For example, images captured in low-light conditions often exhibit poor visibility. This blurry imagery can severely impact the accuracy and efficiency of event assessment when retrieving surveillance information. To address this issue, diffusion models are currently widely used to reconstruct surveillance images to enhance image clarity. However, traditional diffusion models require high computational resources and time, increasing the computational overhead of image reconstruction and reducing reconstruction efficiency. Summary of the Invention

[0003] In order to solve the above-mentioned problems existing in the prior art, the present application provides an image reconstruction method, device and medium based on a potential diffusion model.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides an image reconstruction method based on a latent diffusion model, comprising:

[0006] Get the data source;

[0007] performing standardization processing on the data source to obtain a standardized image;

[0008] Inputting the standardized image into an encoder network to obtain a mean tensor and a standard deviation tensor;

[0009] generating a latent variable based on the mean tensor, the standard deviation tensor, and random noise sampled from a standard normal distribution;

[0010] A cosine scheduling strategy is used to dynamically control the noise addition intensity, and the latent variables at each time step are forward diffused until they converge to a standard normal distribution, thereby obtaining updated latent variables and damaged images.

[0011] Inputting the damaged image into a conditional encoder to obtain conditional information;

[0012] Performing a high-dimensional space projection operation on the conditional information to generate a conditional embedding;

[0013] Using a cross-attention mechanism to align and interact features between the conditional embedding and the updated latent variable to obtain a conditionally enhanced latent variable;

[0014] Encode the time step to obtain a time step code;

[0015] Inputting the time step encoding, conditionally enhanced latent variables, and conditional embedding into a noise prediction network to obtain prediction noise;

[0016] The denoising diffusion implicit model is used to perform inverse diffusion based on the predicted noise to obtain the reconstructed image.

[0017] Optionally, performing standardization processing on the data source to obtain a standardized image includes:

[0018] Loading and parsing data from the data source to obtain a data tensor;

[0019] Performing spatial normalization processing on the data tensor to obtain an image tensor;

[0020] Perform pixel value normalization processing on the image tensor to obtain the normalized image.

[0021] Optionally, the encoder network is constructed based on an improved ResNet50 architecture; the encoder network includes a ResNet50 backbone network with the global pooling layer removed, stacked residual blocks, adaptive downsampling, a mean branch, and a variance branch.

[0022] Optionally, the normalized image is input into an encoder network to obtain a mean tensor and a standard deviation tensor, including:

[0023] After the standardized image is sequentially input into the ResNet50 backbone network, the stacked residual block and the adaptive downsampling, the mean branch and the variance branch are respectively used for parallel processing to obtain the mean tensor and the standard deviation tensor.

[0024] Optionally, a cosine scheduling strategy is used to dynamically control the noise addition intensity, and forward diffusion is performed on the latent variable at each time step until it converges to a standard normal distribution. The process of obtaining the updated latent variable includes:

[0025] The cosine scheduling strategy is used to dynamically control the noise addition intensity and obtain the noise attenuation coefficient;

[0026] Based on the noise attenuation coefficient, the discrete diffusion formula is used Iteratively update the latent variables at each time step until the iteratively updated latent variables converge to the standard normal distribution, and then obtain the updated latent variables;

[0027] Where, is the time step The latent variables, is the time step The latent variables, is the time step The noise attenuation coefficient, is the time step The noise added when , is the standard normal distribution, represents unit variance.

[0028] Optionally, a cross-attention mechanism is used to align and interact features between the conditional embedding and the updated latent variable to obtain a conditionally enhanced latent variable, including:

[0029] Map the updated latent variables to the query vector via a learnable projection matrix;

[0030] The conditional embedding generates keys and values ​​through an independent projection matrix;

[0031] Perform dot product attention calculation on the query vector, key and value after scaling to obtain the attention output;

[0032] The attention output is weighted by a learnable scaling factor and then residually connected to the updated latent variable to obtain a conditionally enhanced latent variable.

[0033] Optionally, the time step is encoded to obtain a time step encoding, including:

[0034] The time step is dynamically encoded using sinusoidal position embedding to obtain the time step encoding; wherein the mapping function of sinusoidal position embedding is expressed as:

[0035] ;

[0036] Where, For channel Medium time step The encoding, is the encoding dimension, is the channel index, .

[0037] Optionally, the noise prediction network is constructed based on an improved UNet architecture; the noise prediction network is a multi-level network consisting of a symmetrical downsampling-upsampling structure; each level includes a residual block and an attention mechanism.

[0038] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the image reconstruction method based on the potential diffusion model provided above.

[0039] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image reconstruction method based on the potential diffusion model provided above.

[0040] According to the specific embodiments provided in this application, this application has the following technical effects:

[0041] This application provides an image reconstruction method, device, and medium based on a latent diffusion model. Leveraging the latent diffusion model's efficient denoising and image restoration capabilities, combined with an encoder network, the method maps high-dimensional image data into a low-dimensional latent space and performs diffusion processing within that space. This method not only reduces computational overhead but also improves image reconstruction efficiency. Furthermore, through the sequential application of a denoising autoencoder (an encoder network and a conditional encoder), it preserves important details of the original image, thereby improving image reconstruction quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 A flowchart of an image reconstruction method based on a latent diffusion model provided in one embodiment of the present application;

[0044] Figure 2 A schematic diagram of an implementation flow of an image reconstruction method based on a potential diffusion model provided in one embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0048] In an exemplary embodiment, the present application provides an image reconstruction method based on a potential diffusion model, which is executed by a computer device. Specifically, the method can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is described by taking the application of the method to a server as an example. Figure 1 As shown, the method includes:

[0049] Step 100: Acquire a data source, wherein an image, a video frame, or a video can be provided as the data source, or an original image or video file read from a storage device can be used as the data source.

[0050] Step 101: normalize the data source to obtain a standardized image.

[0051] Step 102: Input the normalized image into the encoder network to obtain a mean tensor and a standard deviation tensor.

[0052] Step 103 : Generate latent variables based on the mean tensor, the standard deviation tensor, and random noise sampled from a standard normal distribution.

[0053] In step 104 , a cosine scheduling strategy is used to dynamically control the noise addition intensity, and forward diffusion is performed on the latent variables at each time step until they converge to a standard normal distribution, thereby obtaining updated latent variables and a damaged image.

[0054] Step 105: Input the damaged image into a conditional encoder to obtain conditional information.

[0055] Step 106: Perform a high-dimensional space projection operation on the conditional information to generate a conditional embedding.

[0056] In step 107, a cross-attention mechanism is used to align and interact features between the conditional embedding and the updated latent variable to obtain the conditionally enhanced latent variable.

[0057] Step 108: Encode the time step to obtain a time step code.

[0058] Step 109 : The time step encoding, the conditionally enhanced latent variables, and the condition are embedded into the noise prediction network to obtain the predicted noise.

[0059] Step 110 : Using a denoising diffusion implicit model, perform inverse diffusion based on the predicted noise to obtain a reconstructed image.

[0060] Based on the description of steps 100 to 110 above, the implementation architecture of the method provided by the present application can include two major parts: providing a data source and image reconstruction. Among them, the image reconstruction part can also include diffusion in the latent space (corresponding to steps 101 to 104), condition control (corresponding to steps 105 to 107), inverse diffusion training and image reconstruction (corresponding to steps 108 to 110). Based on this, the present application uses efficient denoising and image restoration capabilities, combined with an encoder network to map high-dimensional image data to a low-dimensional latent space for diffusion processing, thereby significantly improving the clarity and detail of the image while reducing computational overhead and maintaining lightweight. Therefore, the present application can not only effectively improve image blurring problems caused by fog, low light or aging equipment, but also maintain high-quality output in complex scenes. It further enhances contrast and edge features through image post-processing technology, making the image easier to analyze and identify.

[0061] In another exemplary embodiment of the present application, in order to unify the spatial size of the input data and adapt to the model requirements, step 100 provided above in the present application can be replaced by the following steps 200 to 202.

[0062] Step 200: Load and parse the data source to obtain a data tensor. For example, read an image or video file from a storage device and convert it into a processable digital format. (a) For an image file, it is decoded into a three-dimensional array with a shape of height ( )×width( ) × RGB color channels (3), the value range of each pixel is an integer from 0 to 255 (uint8 type), and the resulting data tensor is (b) For a video file, it is decomposed into an image sequence frame by frame, and the format of each frame is made consistent with the static image in (a). The resulting data tensor is .

[0063] Step 201: Perform spatial normalization on the data tensor to obtain an image tensor. For example, the image is scaled to a fixed resolution using a bicubic interpolation algorithm to obtain an image tensor. The obtained image tensor is represented as:

[0064] Where, is the image tensor, is a function used to adjust the image size. Scale to a fixed resolution required by the model and store the scaled result in variable.

[0065] The bicubic interpolation algorithm calculates new pixel values ​​based on the weighted average of surrounding pixels. It can preserve edge details while smoothing the transition and avoid jagged or blurring problems caused by scaling.

[0066] Step 202: Normalize the pixel values ​​of the image tensor to obtain a standardized image. This step is mainly to achieve pixel value normalization after the size adjustment is completed in step 201. For example, linearly map the pixel values ​​from the integer range [0, 255] of the uint8 type to the floating point range [-1, 1] of the float32 type to obtain , This is a normalized image. The specific operation of this formula is: first divide the pixel value by 127.5 to scale it to the interval [0, 2], and then subtract 1.0 to align the value center to [-1, 1].

[0067] Finally, the preprocessed data (i.e., the standardized image) obtained based on the above steps 200 to 202 is input into the encoder network with a uniform size and value range to ensure the efficiency and consistency of the training and inference processes.

[0068] In another exemplary embodiment of the present application, in order to perform latent space mapping, the encoder network used in the above step 102 of the present application is essentially a variational autoencoder (VAE), which is built based on the improved ResNet50 architecture and realizes the mapping of input data to latent space through customized design. ) First, the ResNet50 backbone network with the global pooling layer removed is used. After stacking residual blocks (ResBlock) and adaptive downsampling, the network is divided into two branches (mean branch and variance branch) for parallel processing. The mean branch uses an independent 1×1 convolution layer to compress the number of channels and generate a mean tensor. , the variance branch generates the standard deviation tensor through an independent 1×1 convolution layer .

[0069] Combined with the above description of this embodiment, step 103 mainly samples latent variables from the probability distribution output by the encoder through the reparameterization technique to ensure that the gradient can be propagated back to the network parameters through random nodes. Based on this, the latent variable is the representation of the input data in the latent space, which is:

[0070] .

[0071] Among them, the mean tensor Represents the central position of the latent space distribution (that is, the central position of the latent variable). Standard deviation tensor Represents the distribution range of the latent variable (i.e., the uncertainty range of the latent variable). From the standard normal distribution Random noise sampled in . is an element-by-element multiplication, that is, the standard deviation tensor and random noise Multiply each corresponding element of . is the latent variable. The latent variable of the output The structured features of the input data are preserved in a low-dimensional latent space, providing a compact and optimizable implicit representation for the subsequent diffusion and reconstruction processes.

[0072] In another exemplary embodiment of the present application, the training details of the encoder network used in the above step 102 of the present application are described to achieve coordinated optimization of latent space mapping and data generation capabilities.

[0073] The mean tensor computed by reparameterization and the standard deviation tensor Input them into the variational autoencoder respectively to construct the loss function, which is:

[0074] .

[0075] in, is the loss function value, To rebuild the losses, is the KL divergence term. Among them, the reconstruction loss reconstructs the image through the decoder D Compared with the original input (i.e. normalized image ), constraining feature fidelity. The KL divergence term forces the latent variable distribution to approach the standard normal distribution to avoid overfitting. The reconstruction loss and KL divergence term are adjusted by hyperparameters Dynamic weighting ensures that the latent space complexity is limited while maintaining high reconstruction accuracy. represents the total dimension of the latent variables, Indicates the The mean tensor of the n-dimensional latent variables, Indicates the The standard deviation tensor of the n-dimensional latent variables.

[0076] The optimization process uses the AdamW optimizer (learning rate ), weight decay Its decoupled weight decay mechanism effectively suppresses the redundancy of encoder network parameters and improves the generalization performance by explicitly separating the regularization term and the gradient update path. At the same time, the adaptive moment estimation strategy accelerates the convergence of the encoder network, ultimately achieving the coordinated optimization of latent space mapping and data generation capabilities.

[0077] In another exemplary embodiment of the present application, in order to ensure that the time step latent variables It is possible to quickly approximate isotropic Gaussian noise. In the forward diffusion process, the cosine scheduling strategy (CosineSchedule) is used as the noise scheduling strategy in step 104 to dynamically control the intensity of noise addition. The implementation process includes:

[0078] Step 300: Use the cosine scheduling strategy to dynamically control the noise addition intensity to achieve progressive noise addition. The noise attenuation coefficient is defined as:

[0079] Where, is the time step Noise attenuation coefficient.

[0080] This scheduling strategy slowly reduces the noise attenuation coefficient in the early stage of diffusion. , in order to retain more original information, and accelerate the decay in the later stage to approach the standard Gaussian distribution, which is more in line with the natural degradation law of data than linear scheduling.

[0081] Step 301: Based on the noise attenuation coefficient, the discrete diffusion formula is used. The latent variables of each time step are iteratively updated until the iteratively updated latent variables converge to the standard normal distribution, and the updated latent variables are obtained. Weighted retention of previous state information, Controls the noise injection intensity for the current step.

[0082] Where, is the time step The latent variables, is the time step The latent variables, is the time step The noise added when , is the standard normal distribution, represents unit variance.

[0083] Based on the description of step 301 above, the method of verifying whether the iteratively updated latent variables converge to the standard normal distribution (i.e., the diffusion process) is to check the final time step ( ) latent variables Whether it converges to the standard normal distribution to ensure the mathematical completeness of the forward diffusion. The empirical mean of the independent samples verifies the central symmetry of the distribution. The fact that the empirical mean approaches zero indicates that the noise scheduling does not introduce systematic bias and the latent variable distribution is symmetric around the origin. The empirical mean is expressed as:

[0084] .

[0085] Where, For the The final time step of independent samples The potential variables of time.

[0086] At the same time, the independence and unit variance characteristics of each dimension are evaluated based on the empirical covariance matrix to verify whether there is redundant correlation between the dimensions. The empirical covariance matrix is ​​expressed as:

[0087] .

[0088] In another exemplary embodiment of the present application, in order to obtain the potential features of the condition information c, in the condition control module, the condition encoder uses the encoder network E that shares parameters with the VAE to perform isomorphic mapping on the condition information c to obtain the condition information. For example, the encoder network E is used to perform isomorphic mapping on the damaged image in the mask restoration task. Perform feature extraction and generate conditional information aligned with the main latent space through isomorphic mapping, which is recorded as: Where, is the encoder network E.

[0089] Based on this, in order to align the conditional information with the latent variables, the conditional fusion mechanism adopted in this application is: Perform a high-dimensional space projection operation to generate conditional embeddings aligned with the dimensions of the latent variables , to eliminate the dimensionality difference between the main network and the conditional encoder. Then, in order to embed the conditional with the current latent variable (i.e. time step latent variables ) and uses a cross-attention mechanism to align and interact with features. Based on this, the implementation process of step 107 can be replaced by the following steps 400 to 403.

[0090] Step 400: Update the latent variables through the learnable projection matrix Mapped to a query vector, we have: .in, Represents the query vector.

[0091] Step 401: while implementing step 400, embed the conditional data through independent projection matrices. Generate keys and values, with: , .in, Indicates the key, Represents a value.

[0092] Step 402: query vector ,key Sum After scaling and performing dot product attention calculation, we get the attention output: .in, represents the attention output, Represents the softmax function.

[0093] In step 403, the attention output is weighted by the learnable scaling factor and then residually connected with the updated latent variable to obtain the conditionally enhanced latent variable. The conditionally enhanced latent variable is expressed as:

[0094] .

[0095] Where, represents the latent variable for conditional reinforcement, represents a learnable scaling factor.

[0096] Based on the above description of this embodiment, the present application dynamically adjusts the conditional injection strength through a gating mechanism, weakens the conditional influence in the early stage of training to avoid mode collapse, and gradually enhances semantic guidance as the network is optimized.

[0097] In another exemplary embodiment of the present application, in the conditional inverse diffusion process, the dynamic encoding of the time step is implemented by sinusoidal positional encoding, and its mapping function is expressed as:

[0098] .

[0099] Where, For channel Medium time step The encoding, is the encoding dimension, is the channel index, .

[0100] This encoding method converts discrete time steps into discrete time steps through an exponentially decaying frequency basis function. It is converted into a continuous high-dimensional vector, injecting time series perception into the noise prediction network, so that it can dynamically adapt to the feature learning requirements of different noise stages.

[0101] In another exemplary embodiment of the present application, in the inverse diffusion process, the noise prediction network in step 109 uses an improved UNet architecture to implement noise prediction. The noise prediction network input is composed of the current noisy latent variable (i.e., the conditionally enhanced latent variable ), conditional embedding and time step encoding These three are fused through feature concatenation and adaptive layer normalization (AdaGN). Modulation parameters are generated by linear projection , to dynamically adjust the convolutional layer weights, Dependent time step Normalized characteristic variance of the modulation, Dependent time step Modulated normalized feature mean. Conditional embedding The downsampling and upsampling modules are injected through cross-scale jump connections to achieve multi-granularity conditional constraints.

[0102] The UNet architecture consists of a multi-layered network structure with a symmetrical downsampling-upsampling structure. Each layer includes a residual block (ResBlock) and an attention mechanism (Attention Block). Residual connections alleviate the vanishing gradient problem by mapping the layers to identical values, while skip connections fuse encoder and decoder features step by step, ensuring accurate reconstruction of high-frequency details.

[0103] The noise prediction network finally outputs the predicted noise , thus achieving the coordinated optimization of accurate estimation of noise distribution and conditional generation control.

[0104] In another exemplary embodiment of the present application, in the reverse diffusion sampling stage, in the above step 110 of the present application, a deterministic denoising diffusion implicit model (DDIM) acceleration strategy is used to replace the traditional Markov chain update method to achieve efficient denoising. This process can be described as follows:

[0105] Noise attenuation coefficient generated based on cosine scheduling , each time step is based on the current latent variable , predict noise And scheduling parameters, according to the deterministic formula Iteratively update the latent variables. Among them, the first term of the deterministic formula The second term corrects the current estimated data mean by the noise residual. Inject deterministic noise to maintain trajectory smoothness. is the time step Noise attenuation coefficient.

[0106] By predicting the noise and the noise attenuation coefficient of the current time step , calculate the noisy data of the previous time step (i.e. The whole process gradually removes noise and finally completes the image reconstruction.

[0107] In summary, the implementation process of the image reconstruction method based on the potential diffusion model provided by this application is as follows: Figure 2 As shown in FIG, it utilizes potential space diffusion, denoising reconstruction and conditional control to complete the processing of images, video frames or videos in the process of potential space diffusion, which can not only reduce the computational cost but also ensure high-quality image reconstruction capabilities.

[0108] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store image reconstruction data based on a potential diffusion model. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for image reconstruction based on a potential diffusion model is implemented.

[0109] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0110] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0111] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0112] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0113] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0114] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (RRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0115] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0116] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An image reconstruction method based on a latent diffusion model, characterized in that: include: Get the data source; performing standardization processing on the data source to obtain a standardized image; Inputting the standardized image into an encoder network to obtain a mean tensor and a standard deviation tensor; generating a latent variable based on the mean tensor, the standard deviation tensor, and random noise sampled from a standard normal distribution; A cosine scheduling strategy is used to dynamically control the noise addition intensity, and the latent variables at each time step are forward diffused until they converge to a standard normal distribution, thereby obtaining updated latent variables and damaged images. Inputting the damaged image into a conditional encoder to obtain conditional information; Performing a high-dimensional space projection operation on the conditional information to generate a conditional embedding; Using a cross attention mechanism to align and interact features between the conditional embedding and the updated latent variable to obtain a conditionally enhanced latent variable, including: mapping the updated latent variable to a query vector through a learnable projection matrix; generating a key and a value from the conditional embedding through an independent projection matrix; performing a dot product attention calculation on the query vector, the key, and the value after scaling to obtain an attention output; and performing a residual connection on the attention output with the updated latent variable after weighting by a learnable scaling factor to obtain the conditionally enhanced latent variable. Encode the time step to obtain a time step code; Inputting the time step encoding, conditionally enhanced latent variables, and conditional embedding into a noise prediction network to obtain prediction noise; Adopting the denoising diffusion implicit model, inverse diffusion is performed based on the predicted noise to obtain the reconstructed image; The cosine scheduling strategy is used to dynamically control the noise addition intensity, and the latent variables at each time step are forward diffused until they converge to the standard normal distribution. The process of obtaining the updated latent variables includes: The cosine scheduling strategy is used to dynamically control the noise addition intensity and obtain the noise attenuation coefficient; Based on the noise attenuation coefficient, the discrete diffusion formula is used Iteratively update the latent variables at each time step until the iteratively updated latent variables converge to the standard normal distribution, and then obtain the updated latent variables; Where, is the time step The latent variables, is the time step The latent variables, is the time step The noise attenuation coefficient, is the time step The noise added when , is the standard normal distribution, represents unit variance.

2. The image reconstruction method based on the latent diffusion model according to claim 1, characterized in that: Standardizing the data source to obtain a standardized image includes: Loading and parsing data from the data source to obtain a data tensor; Performing spatial normalization processing on the data tensor to obtain an image tensor; Perform pixel value normalization processing on the image tensor to obtain the normalized image.

3. The image reconstruction method based on the latent diffusion model according to claim 1, characterized in that: The encoder network is constructed based on an improved ResNet50 architecture; the encoder network includes a ResNet50 backbone network with the global pooling layer removed, stacked residual blocks, adaptive downsampling, mean branch and variance branch.

4. The image reconstruction method based on the latent diffusion model according to claim 3, characterized in that: The standardized image is input into the encoder network to obtain the mean tensor and standard deviation tensor, including: After the standardized image is sequentially input into the ResNet50 backbone network, the stacked residual block and the adaptive downsampling, the mean branch and the variance branch are respectively used for parallel processing to obtain the mean tensor and the standard deviation tensor.

5. The image reconstruction method based on the latent diffusion model according to claim 1, characterized in that: Encode the time step to obtain the time step encoding, including: The time step is dynamically encoded using sinusoidal position embedding to obtain the time step encoding; wherein the mapping function of sinusoidal position embedding is expressed as: ； Where, For channel Medium time step The encoding, is the encoding dimension, is the channel index, .

6. The image reconstruction method based on the latent diffusion model according to claim 1, characterized in that: The noise prediction network is built based on an improved UNet architecture; the noise prediction network is a multi-level network consisting of a symmetrical downsampling-upsampling structure; each level contains a residual block and an attention mechanism.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the image reconstruction method based on the potential diffusion model according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image reconstruction method based on the latent diffusion model according to any one of claims 1 to 6 is implemented.

Images