End-to-end diffusion type hierarchical probability image reconstruction method and system, medium and product

Abstract

The invention discloses an end-to-end diffusion type hierarchical probability image reconstruction method and system, a medium and a product, and the method comprises the steps: reconstructing according to projection data to obtain an initial blurred image containing a rough contour of an imaged object, and segmenting the initial blurred image into image blocks with the same size in an image domain; rearranging all the image blocks into a * b * c three-dimensional tensor, constructing a graph structure by taking each image block as a node and taking the similarity between the image blocks as an edge, performing message passing and feature aggregation on the graph structure by utilizing a graph convolutional network, and taking updated node features as inter-layer features; and constructing an a * b * 2c empty tensor, for each layer of image in the c direction, reconstructing an a * b * 2c tensor in a layered manner by using a diffusion denoising probability model, adding the inter-layered features in each diffusion denoising of the diffusion denoising probability model, and rearranging and combining the obtained c a * b tensors into a finally reconstructed target image, the reconstruction method can realize high-quality image reconstruction under the conditions of low signal-to-noise ratio and sparse projection data, and is suitable for an embedded hardware platform and real-time or near-real-time imaging application.

Description

Technical Field

[0001] This invention relates to the field of imaging technology, and in particular to an end-to-end diffusion-based layered probabilistic image reconstruction method, system, medium, and product. Background Technology

[0002] Imaging technology has wide applications in fields such as medicine (e.g., CT, DR), autonomous driving, and atmospheric environment. However, in practical applications, due to factors such as limited photon source intensity, low detection efficiency, and sparse projection data, the acquired projection data often has a low signal-to-noise ratio and limited information content, making it difficult for traditional reconstruction methods (such as algebraic iterative reconstruction and probabilistic reconstruction) to obtain clear, high-resolution reconstructed images.

[0003] While some image reconstruction methods based on deep learning or probabilistic models have attempted to improve image quality, these methods typically suffer from the following problems: 1. High computational complexity, making it difficult to run in real time on embedded or edge computing devices; 2. Sensitive to noise, especially under extremely low signal-to-noise ratio conditions, performance degrades significantly; 3. The lack of effective utilization of structural features between images leads to loss of details or severe artifacts in the reconstructed image.

[0004] Therefore, there is an urgent need for a new image reconstruction method and system that can maintain high reconstruction quality and has efficient computing power. Summary of the Invention

[0005] Based on the technical problems existing in the background technology, this invention proposes an end-to-end diffusion-based hierarchical probabilistic image reconstruction method, system, medium and product, which can achieve high-quality image reconstruction under low signal-to-noise ratio and sparse projection data conditions.

[0006] This invention proposes an end-to-end diffusion-based hierarchical probabilistic image reconstruction method, comprising: An initial blurred image containing the approximate outline of the imaged object is reconstructed based on the projection data obtained by the imaging system, and in the image domain, the initial blurred image is divided into image blocks of the same size. All image patches are rearranged into a three-dimensional tensor of a×b×c. Each image patch is used as a node, and the similarity between image patches is used as an edge to construct a graph structure. A graph convolutional network is used to perform message passing and feature aggregation on the graph structure. The updated node features are used as inter-layer features, where a×b is the length and height of the image patch, and c is the number of sub-image patches. Construct an empty tensor of a×b×2c. For each layer of the image in the c direction, use a diffusion denoising probability model to reconstruct the a×b×2c tensor layer by layer. In each diffusion denoising step of the diffusion denoising probability model, add the inter-layer features. Rearrange and combine the obtained c a×b tensors to form the final reconstructed target image.

[0007] Furthermore, the process of constructing the target image is as follows: The a×b×2c tensor is reconstructed hierarchically using a diffusion denoising probability model, where the first c channels are used for the reconstruction of the initial blurred image and the last c channels are used for the reconstruction of the target image. The inter-layer features are added to each diffusion denoising step of the diffusion denoising probability model, and a “repeatedly adding noise-denoising” process is used until the diffusion denoising probability model converges. The last c a×b images are rearranged and combined to form the final reconstructed target image.

[0008] Furthermore, the a×b×2c tensor is reconstructed hierarchically using a diffusion denoising probability model, and the reconstruction is divided into 2c computational processes for parallel processing, with each parallel computational process processing one a×b tensor.

[0009] Furthermore, in the process of "repeatedly adding noise and denoising" until the diffusion denoising probability model converges, a minimum and a maximum number of iteration steps are set. After completing the minimum number of iteration steps, the probability distribution difference between forward and backward Gaussian noise is obtained by calculating the KL divergence. The iteration stops when the probability distribution difference is less than or equal to the set difference threshold or when the maximum number of iteration steps is reached.

[0010] Furthermore, the step of reconstructing an initial blurred image containing the approximate outline of the imaged object based on the projection data specifically involves reconstructing the initial blurred image using an algebraic iterative reconstruction method based on the projection data.

[0011] Furthermore, the image reconstruction method is used to run on embedded or edge computing devices.

[0012] Furthermore, the diffusion denoising probability model is a dual-channel conditional denoising network, which receives the following three inputs simultaneously in each denoising step: 1) The current noisy tensor a×b×2c; 2) Inter-layer features, which are mapped into channel attention vectors through a feature fusion module, and used to modulate the attention of the conditional denoising network to the preceding and following channels; 3) A preset channel mask is used to explicitly indicate whether the conditional denoising network focuses on the "reconstruction consistency" of the first c channels or the "generation authenticity" of the last c channels in the current denoising step.

[0013] A computer system includes a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the method described above.

[0014] A computer-readable storage medium storing a plurality of classification programs, the plurality of classification programs being invoked by a processor to execute the method described above.

[0015] A computer program product includes a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method described above.

[0016] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk. The advantages of the end-to-end diffusion-based hierarchical probabilistic image reconstruction method, system, medium, and product provided by this invention are as follows: by introducing a diffusion probability model and a hierarchical feature extraction mechanism, high-quality image reconstruction can be achieved under conditions of low signal-to-noise ratio and sparse projection data. At the same time, through parallel design and hardware adaptation, it has high computational efficiency and is suitable for embedded hardware platforms and real-time or near-real-time imaging applications. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0019] like Figure 1 As shown, the present invention proposes an end-to-end diffusion-based hierarchical probabilistic image reconstruction method, comprising: Step 1: Reconstruct an initial blurred image containing the approximate outline of the imaged object based on the projection data, and then divide the initial blurred image into image blocks of the same size in the image domain. Step 2: Rearrange all image blocks into a three-dimensional tensor of a×b×c. Construct a graph structure with each image block as a node and the similarity between image blocks as edges. Use a graph convolutional network to perform message passing and feature aggregation on the graph structure. Use the updated node features as inter-layer features, where a×b is the length and height of the image block, and c is the number of sub-image blocks, also known as the number of channels. These image patches represent local regions of the image, and each image patch corresponds to one channel of the tensor; Step 3: Construct an empty tensor of a×b×2c. For each layer of the image in the c direction, use a diffusion denoising probability model to reconstruct the a×b×2c tensor layer by layer. In each diffusion denoising step of the diffusion denoising probability model, add the inter-layer features. Rearrange and combine the obtained c a×b tensors into the final reconstructed target image.

[0020] According to steps one to three, this embodiment can achieve high-quality image reconstruction under low signal-to-noise ratio and sparse projection data conditions by introducing a diffusion probability model and a hierarchical feature extraction mechanism. At the same time, through parallel design and hardware adaptation, it has high computational efficiency and is suitable for embedded hardware platforms and real-time or near-real-time imaging applications.

[0021] In this embodiment, step two involves rearranging all image blocks into a three-dimensional tensor of size a×b×c. Each image block is used as a node, and the similarity between image blocks is used as edges to construct a graph structure. A graph convolutional network is then used to perform message passing and feature aggregation on this graph structure. The relationships between image blocks are explicitly modeled by updating node features, and the updated node features are used as inter-layer features. Here, a×b represents the length and height of the image block, and c represents the number of sub-image blocks. Specifically: (a1) The a×b×2c tensor is reconstructed hierarchically using a diffusion denoising probability model, where the first c channels are used for the reconstruction of the initial blurred image and the last c channels are used for the reconstruction of the target image. (a2) Construct a graph structure with each image patch as a node and the similarity between image patches as edges. Use a graph convolutional network to perform message passing and feature aggregation on the graph structure and use the updated node features as inter-layer features. In the graph structure, each image patch is considered a node (i.e., the node's features are the pixel values ​​or extracted features of the image patch). Edges are constructed based on the similarity between image patches. Similarity is typically determined by calculating the feature distance between image patches (such as Euclidean distance or cosine similarity). Specific construction methods may include: k-nearest neighbor (k-NN) graph: For each node, select the k most similar nodes to establish edges. Thresholding method: If the similarity between two nodes exceeds a preset threshold, an edge is established.

[0022] The graph structure is represented by an adjacency matrix A, where Represents a node and Does an edge exist between them (or the weight of the edge, based on similarity)? Typically, the adjacency matrix is ​​normalized to facilitate graph convolution operations.

[0023] This embodiment constructs a graph structure and then performs message passing and feature aggregation through a graph convolutional network because (b1) to (b3): (b1) Capturing non-local similarity: There may be semantic or structural similarities (such as repeated textures or edges) between image patches, which cannot be fully captured by local convolution. Graph structures allow the model to directly model long-range dependencies between image patches.

[0024] (b2) Enhanced Feature Representation: Through graph convolutional networks, nodes (image patches) can aggregate information from neighboring nodes to obtain richer contextual features. This helps maintain overall structural consistency and reduce artifacts during reconstruction.

[0025] (b3) Adapting to irregular structures: The initial blurred image may contain complex or irregular object outlines. Graph structures can flexibly represent these relationships without being limited by regular grids.

[0026] The graph convolutional network in this embodiment performs message passing and feature aggregation on a graph structure, as detailed in (c1) to (c3): (c1) Message Passing: In each layer, each node receives "messages" from its direct neighbors. A message is typically a feature vector of a neighboring node, possibly transformed by a weight matrix. Message passing can be represented as: ; in, It is a node In the The message received by the layer It is a node The neighborhood group, It is a node In the Features of the layer It is a learnable weight matrix.

[0027] (c2) Feature aggregation: Nodes combine received messages with their own features to update their own features. Common aggregation methods include: ; in, The normalized adjacency matrix, It is the first The feature matrix of all nodes in the layer, It is a node In the Characteristics of the layer.

[0028] (c3) Multi-layer stacking; Through multiple layers of a graph convolutional neural network (GCN), messages propagate multi-hops within the graph structure, enabling each node to capture broader contextual information. Ultimately, the updated node features, termed inter-layer features, encode the similarity and structural relationships between image patches.

[0029] The inter-layer features obtained through (c1) to (c3) are used as conditional information for the subsequent diffusion denoising probabilistic model. These features are typically reshaped or mapped to a format suitable for the input of the diffusion model (e.g., converted into channel attention vectors via a feature fusion module).

[0030] (a3) The inter-layer features are added to each diffusion denoising step of the diffusion denoising probability model, and the process of "repeatedly adding noise and denoising" is adopted until the diffusion denoising probability model converges. (a4) The last c a×b obtained are rearranged and combined to form the final reconstructed target image.

[0031] The core innovation of the diffusion denoising probability model in this embodiment lies in a dual-channel conditional denoising network. In each denoising step, the network simultaneously receives the following three inputs: 1) the currently denoised a×b×2c tensor; 2) the inter-layer features, which are mapped into channel attention vectors through a feature fusion module to modulate the network's attention to the preceding and following channels; and 3) a preset channel mask.

[0032] Regarding the three inputs above: 1) The current noisy a×b×2c tensor; this is an intermediate state in the diffusion process, representing the noisy image at the current step. The first c channels of the tensor correspond to the initial blurred image patch, and the last c channels correspond to the target image patch (the image being reconstructed).

[0033] 2) The inter-layer features mentioned above: the features obtained from the graph convolutional neural network are mapped into channel attention vectors through the feature fusion module.

[0034] 3) A preset channel mask: A binary or soft mask used to indicate whether the network should currently focus on the "reconstruction consistency" of the first c channel (blurred image) or the "generation realism" of the last c channel (target image).

[0035] The diffusion denoising probability model in this embodiment increases the correlation between different layers through the above three inputs, and has the advantages of multi-source conditional fusion, stable training and generation, and flexibility: Multi-source conditional fusion: Layered features provide global context between image patches, which helps maintain structural consistency; channel masks allow dynamic control of reconstruction strategies, balancing fidelity to the input (reconstruction consistency) and generation of high-quality details (generation realism). Stable training and generation: Conditional information guides the denoising process, preventing the diffusion model from deviating from the target and improving the accuracy and visual quality of reconstruction; Flexibility: The channel mask can be adjusted according to task requirements, for example, emphasizing reconstruction consistency in the early denoising steps and emphasizing generation realism in the later steps.

[0036] The diffusion denoising probability model processes the above three inputs in steps (d1) to (d3): (d1) Current noisy tensor: As the main input of the denoising network, it is usually extracted by convolutional layers to extract local features.

[0037] (d2) Features between layers; First, it is mapped to a channel attention vector with a size of 2c (matching the number of channels of the tensor) through a feature fusion module (e.g., a fully connected layer or attention mechanism). This attention vector is used to modulate the channel importance of the tensor: each channel of the tensor is weighted through a channel attention mechanism (such as squeeze-and-excitation in SENet), with the formula roughly as follows: ; in, For the weighted tensor, It is the current noisy tensor. It is the sigmoid function. These are learnable parameters. It is a feature between layers.

[0038] (d3) Channel mask: The mask may be a vector with the same dimensions as the tensor channels (e.g., the first c bits are 1 and the last c bits are 0, or vice versa), or a learnable parameter. Masks are combined with attention vectors, for example, through element-wise multiplication, to emphasize specific channels. For instance, when the mask indicates "reconstructing consistency," the network boosts the weights of the first c channels; when it indicates "generating realism," it boosts the weights of the last c channels.

[0039] The diffusion denoising probabilistic model processes the three inputs through steps (d1) to (d3), and the resulting features and applications are as follows: Denoising Output: After multiple denoising steps (usually tens to hundreds of steps), the diffusion model outputs a denoised a×b×2c tensor. The last c channels of this tensor are the reconstructed target image patch; Feature usage: The intermediate features obtained in each denoising step are used to progressively reduce noise and refine image details. Finally, c a×b image blocks are rearranged and combined (by stitching or overlapping and fusing) into a complete target image; Overall goal: The resulting reconstructed images are high-resolution, rich in detail, and structurally consistent, making them suitable for applications requiring precise images, such as medical imaging and industrial inspection.

[0040] It should be noted that in tensor reconstruction using the diffusion denoising probability model, the tensor reconstruction process can be processed in parallel, with each a×b tensor processed by 2c processing units to improve computation speed and reduce storage overhead.

[0041] In addition, this embodiment also provides a system for implementing the above method, which has the ability to operate efficiently in embedded hardware, including: Image preprocessing unit, used for image segmentation and tensor construction; 2c parallel processing units are used to perform the reconstruction of each a×b tensor in parallel; The image integration and fusion unit is used for stitching and fusion of the final image.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An end-to-end diffusion-based hierarchical probabilistic image reconstruction method, characterized in that, include: An initial blurred image containing the approximate outline of the imaged object is reconstructed based on the projection data obtained by the imaging system, and in the image domain, the initial blurred image is divided into image blocks of the same size. All image patches are rearranged into a three-dimensional tensor of a×b×c. Each image patch is used as a node, and the similarity between image patches is used as an edge to construct a graph structure. A graph convolutional network is used to perform message passing and feature aggregation on the graph structure. The updated node features are used as inter-layer features, where a×b is the length and height of the image patch, and c is the number of sub-image patches. Construct an empty tensor of a×b×2c. For each layer of the image in the c direction, use a diffusion denoising probability model to reconstruct the a×b×2c tensor layer by layer. In each diffusion denoising step of the diffusion denoising probability model, add the inter-layer features. Rearrange and combine the obtained c a×b tensors to form the final reconstructed target image.

2. The image reconstruction method according to claim 1, characterized in that, The process of constructing the target image is as follows: The a×b×2c tensor is reconstructed hierarchically using a diffusion denoising probability model, where the first c channels are used for the reconstruction of the initial blurred image and the last c channels are used for the reconstruction of the target image. The inter-layer features are added to each diffusion denoising step of the diffusion denoising probability model, and the "repeated addition of noise-denoising" process is used until the diffusion denoising probability model converges. The last c a×b images are rearranged and combined to form the final reconstructed target image.

3. The image reconstruction method according to claim 1, characterized in that, The method uses a diffusion denoising probability model to reconstruct the a×b×2c tensor in layers, dividing the reconstruction into 2c computational processes for parallel processing, with each parallel computational process processing one a×b tensor.

4. The image reconstruction method according to claim 2, characterized in that, In the process of "repeatedly adding noise and denoising" until the diffusion denoising probability model converges, a minimum and a maximum number of iteration steps are set. After the minimum number of iteration steps is completed, the probability distribution difference between forward and backward Gaussian noise is obtained by calculating the KL divergence. The iteration stops when the probability distribution difference is less than or equal to the set difference threshold or when the maximum number of iteration steps is reached.

5. The image reconstruction method according to claim 1, characterized in that, The process of reconstructing an initial blurred image containing the approximate outline of the imaged object based on the projection data obtained from the imaging system specifically involves reconstructing the initial blurred image using an algebraic iterative reconstruction method based on the projection data.

6. The image reconstruction method according to claim 1, characterized in that, The image reconstruction method is used to run on embedded or edge computing devices.

7. The image reconstruction method according to claim 1, characterized in that, The diffusion denoising probability model is a dual-channel conditional denoising network, which receives the following three inputs simultaneously in each denoising step: 1) The current noisy tensor a×b×2c; 2) Inter-layer features, which are mapped into channel attention vectors through a feature fusion module, and used to modulate the attention of the conditional denoising network to the preceding and following channels; 3) A preset channel mask is used to explicitly indicate whether the conditional denoising network focuses on the "reconstruction consistency" of the first c channels or the "generation authenticity" of the last c channels in the current denoising step.

8. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of classification programs, which are used by a processor to execute the method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images