A three-dimensional near-infrared spectral tomography reconstruction method based on mask prior

Abstract

The application discloses a three-dimensional near-infrared spectral tomography reconstruction method based on a mask prior, and belongs to the field of medical image processing. The method respectively performs multi-scale convolution coding on NIRST optical signals and MRI volume data to obtain features, then embeds structural information of a mask as a prior condition into a feature fusion module, dynamically adjusts spatial weights in the fusion process of multi-scale structural information and optical functional information, so that the network focuses on potential lesion areas and suppresses background noise, and finally reconstructs the optical parameter distribution of the tissue through a three-dimensional decoding network. The method provided by the application can effectively reduce the ill-posedness in the NIRST reconstruction problem, reduce artifact generation, improve the positioning accuracy of the tumor area and the optical property reconstruction quality, and reconstruct the optical property parameter distribution of the biological tissue.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and relates to a method for three-dimensional near-infrared spectral tomography image reconstruction based on mask prior. Background Technology

[0002] Near-infrared spectral tomography (NIRST) is a novel optical imaging diagnostic technique that detects the absorption and scattering characteristics of near-infrared light in the 700-900 nm wavelength range in biological tissues. It inverts optical parameters within the tissue, such as blood oxygen saturation, total hemoglobin concentration, and scattering coefficient, thereby reflecting the physiological state and pathological characteristics of the tissue. NIRST offers advantages such as low cost, non-ionizing nature, and non-invasiveness. It has been widely applied in various medical scenarios, including brain function testing, tissue hemoglobin detection, and early breast cancer detection.

[0003] For NIRST image reconstruction, the problem is highly ill-posed because the number of measurements acquired by the optical detector is much smaller than the number of optical parameters of the biological tissue to be reconstructed. The reconstruction solution may not be unique and is extremely sensitive to measurement noise.

[0004] To reduce ill-posedness in image reconstruction, commonly used reconstruction methods rely primarily on manually designed physical models and mathematical optimizations, including algorithms such as regularization and algebraic iterative reconstruction. Insufficient regularization makes the image susceptible to noise interference and artifacts, while excessive regularization leads to image smoothing and loss of detail. Iterative reconstruction algorithms require multiple iterations for optimization, resulting in high computational complexity and long processing times, making them unsuitable for real-time diagnostic needs. Therefore, this invention proposes a method for reconstructing three-dimensional near-infrared spectral tomography based on a mask-based prior convolutional neural network. Summary of the Invention

[0005] The purpose of infrared spectral tomography is to reconstruct the distribution of optical property parameters of biological tissues by using weak light signals collected on the surface of biological tissues and combining them with the transmission law of light in biological tissues.

[0006] To achieve the above objectives, this invention proposes a three-dimensional near-infrared spectral tomography reconstruction algorithm based on mask priors. This algorithm guides the reconstruction of the optical parameter distribution of biological tissues using MRI volume data and NIRST signals based on the spatial features of the mask, thereby reducing artifacts in the reconstructed images.

[0007] A three-dimensional near-infrared spectral tomography reconstruction method based on mask prior is characterized by a mask-guided attention mechanism, comprising the following steps:

[0008] Step 1: Mask I corresponding to the t-th MRI slice t It can be described as:

[0009] (1)

[0010] Among them, I t (x, y)=0,1 indicates that the pixel (x, y) belongs to the background and tumor regions, respectively, and H and W are the length and width of the slice, respectively.

[0011] Step 2: Statistical slice I t The number of tumor pixels N t :

[0012] (2)

[0013] Step 3: Determine each I t Validity:

[0014] (3)

[0015] Wherein, Valid(I t If )=1, it means that the slice contains a tumor region and is a valid slice. The mask will be used as a conditional input in the attention calculation. Otherwise, it is an invalid slice and will not be used as a conditional input.

[0016] Step 4: Combine NIRST features, MRI image features, and features from the previous layer to obtain the fused feature F0 for each layer:

[0017] (4)

[0018] Where ⊕ represents feature concatenation, F NIRST This indicates that initial features are extracted using 3×3×3 convolution and Leaky-ReLU, and then two branches are used to recover spatial resolution using 2×2×2 upsampling convolution and to extract deep semantic features using 2×2×2 downsampling convolution, respectively. F MRI This indicates that multi-scale MRI structural features are progressively extracted by using two 3×3×3 convolutions and Leaky-ReLU layer by layer, combined with adaptive max pooling downsampling. F prev This indicates features derived from the previous fusion layer, and in the first fusion layer, this module only receives MRI and NIRST features.

[0019] Step 5: Adjust the effective slicing mask using 1×1×1 convolution The number of channels, to obtain I 1 :

[0020] (5)

[0021] Where B is batch size, C is the number of channels, D is the number of slices, and k is the number of segments. 1×1×1This represents a 1×1×1 convolution kernel, * represents the convolution operation, and R(·) represents the Leaky-ReLU activation function.

[0022] Step 6: Employ two consecutive 3×3×3 3D convolutional layers and a LeakyReLU activation function to downsample layer by layer, continuously compressing the spatial resolution and extracting multi-scale structural features:

[0023] (6)

[0024] Where l represents the downsampling index layer, k 3×3×3 P represents a 3D convolution kernel. max This represents a 2×2×2 max pooling operation.

[0025] Step 7: Perform max pooling and average pooling operations on F0, and concatenate it with the mask features. Extract cross-modal fusion features through 3×3×3 convolution and LeakyReLU activation.

[0026] (7)

[0027] In this context, [·] indicates a channel splicing operation.

[0028] Step 8: Generate a spatial attention weight map using the Sigmoid function:

[0029] (8)

[0030] Where σ(·) represents the Sigmoid activation function.

[0031] Step 9: In the attention calculation, a residual form is used to fuse the features after the spatial attention weights of the mask are dynamically adjusted with the features before adjustment:

[0032] (9)

[0033] Where ⊙ represents element-wise multiplication, and F´ represents the output feature.

[0034] Step 10: NIRST Image Reconstruction:

[0035] The fused features are upsampled step by step by the decoder, and skip connections corresponding to the encoder layer are introduced to fuse features, gradually restoring the resolution, and finally outputting the reconstructed result of the optical characteristic parameter distribution. Attached Figure Description

[0036] Figure 1 This is a flowchart of the present invention;

[0037] Figure 2This is a schematic diagram of the network structure of the present invention. Its inputs are NIRST signals, MRI volume data and tumor mask volume data, and its outputs are the distribution and concentration values ​​of optical property parameters.

[0038] Figure 3 This is a diagram of the mask-guided dual-modal fusion attention mechanism of the present invention. Detailed Implementation

[0039] A three-dimensional near-infrared spectral tomography reconstruction method based on mask prior includes the following steps:

[0040] Step 1: Mask I corresponding to the t-th MRI slice t It can be described as:

[0041] (1)

[0042] Among them, I t (x, y)=0,1 indicates that the pixel (x, y) belongs to the background and tumor regions, respectively, and H and W are the length and width of the slice, respectively.

[0043] Step 2: Statistical slice I t The number of tumor pixels N t :

[0044] (2)

[0045] Step 3: Determine each I t Validity:

[0046] (3)

[0047] Wherein, Valid(I t If )=1, it means that the slice contains a tumor region and is a valid slice. The mask will be used as a conditional input in the attention calculation. Otherwise, it is an invalid slice and will not be used as a conditional input.

[0048] Step 4: Combine NIRST features, MRI image features, and features from the previous layer to obtain the fused feature F0 for each layer:

[0049] (4)

[0050] Where ⊕ represents feature concatenation, F NIRST This indicates that initial features are extracted using 3×3×3 convolution and Leaky-ReLU, and then two branches are used to recover spatial resolution using 2×2×2 upsampling convolution and to extract deep semantic features using 2×2×2 downsampling convolution, respectively. F MRIThis indicates that multi-scale MRI structural features are progressively extracted by using two 3×3×3 convolutions and Leaky-ReLU layer by layer, combined with adaptive max pooling downsampling. F prev This indicates features derived from the previous fusion layer, and in the first fusion layer, this module only receives MRI and NIRST features.

[0051] Step 5: Adjust the effective slicing mask using 1×1×1 convolution The number of channels, to obtain I 1 :

[0052] (5)

[0053] Where B is batch size, C is the number of channels, D is the number of slices, and k is the number of segments. 1×1×1 The expression represents a 1×1×1 convolution kernel, * represents a convolution operation, and R(·) represents the Leaky-ReLU activation function. In this embodiment, B=8, C=1, H=128, W=128, and D=144.

[0054] Step 6: Employ two consecutive 3×3×3 3D convolutional layers and a LeakyReLU activation function to downsample layer by layer, continuously compressing the spatial resolution and extracting multi-scale structural features:

[0055] (6)

[0056] Where l represents the downsampling index layer, k 3×3×3 P represents a 3D convolution kernel. max This represents a 2×2×2 max pooling operation.

[0057] Step 7: Perform max pooling and average pooling operations on F0, and concatenate it with the mask features. Extract cross-modal fusion features through 3×3×3 convolution and LeakyReLU activation.

[0058] (7)

[0059] In this context, [·] indicates a channel splicing operation.

[0060] Step 8: Generate a spatial attention weight map using the Sigmoid function:

[0061] (8)

[0062] Where σ(·) represents the Sigmoid activation function.

[0063] Step 9: In the attention calculation, a residual form is used to fuse the features after the spatial attention weights of the mask are dynamically adjusted with the features before adjustment:

[0064] (9)

[0065] Where ⊙ represents element-wise multiplication, and F´ represents the output feature.

[0066] Step 10: NIRST Image Reconstruction:

[0067] The fused features are upsampled step by step by the decoder, and skip connections corresponding to the encoder layers are introduced for feature fusion to gradually restore the resolution, ultimately outputting the reconstructed distribution of optical characteristic parameters. In this embodiment, the final output size is 1×1×128×128×144.

[0068] The above-described three-dimensional near-infrared spectral tomography reconstruction model based on mask prior is the mask prior three-dimensional near-infrared spectral tomography reconstruction method of this invention. This invention not only reconstructs the distribution of optical characteristic parameters with relatively high accuracy but also has high computational efficiency.

Claims

1. A three-dimensional near-infrared spectral tomography reconstruction method based on mask prior, characterized in that, Includes the following steps: Step 1: Mask I corresponding to the t-th MRI slice t Described as: (1)； Among them, I t (x, y)=0,1 indicates that the pixel (x, y) belongs to the background and tumor regions respectively, and H and W are the length and width of the slice respectively; Step 2: Statistical Slice I t The number of tumor pixels N t : (2)； Step 3: Determine the I value of each slice t Validity: (3)；； Wherein, Valid(I t If )=1, it means that the slice contains a tumor region and is a valid slice. The mask will be used as a conditional input in the attention calculation. Otherwise, it is an invalid slice and will not be used as a conditional input. Step 4: Combine NIRST features, MRI image features, and features from the previous layer to obtain the fused feature F0 for each layer: (4)； Where ⊕ represents feature concatenation, F NIRST This indicates that initial features are extracted using 3×3×3 convolution and Leaky-ReLU, and then two branches are used to recover spatial resolution using 2×2×2 upsampling convolution and to extract deep semantic features using 2×2×2 downsampling convolution, respectively. F MRI This indicates that after adjusting the channels using 1×1×1 convolution, two 3×3×3 convolutions and Leaky-ReLU are applied layer by layer, combined with adaptive max pooling downsampling, to progressively extract multi-scale MRI structural features. F prev This indicates features derived from the previous fusion layer, and only MRI and NIRST features are received in the first fusion layer; Step 5: Adjust the effective slicing mask using 1×1×1 convolution. The number of channels, to obtain I 1 : (5)； Where B is batch size, C is the number of channels, D is the number of slices, and k is the number of segments. 1×1×1 This represents a 1×1×1 convolution kernel, * represents the convolution operation, and R(·) represents the Leaky-ReLU activation function; Step Six: Employ two consecutive 3×3×3 three-dimensional convolutional layers and a LeakyReLU activation function to downsample layer by layer, continuously compressing the spatial resolution and extracting multi-scale structural features: (6)； Where l represents the downsampling index layer, k 3×3×3 P represents a 3D convolution kernel. max This represents a 2×2×2 max pooling operation; Step 7: Perform max pooling and average pooling operations on F0, and concatenate it with the mask features. Extract cross-modal fusion features through 3×3×3 convolution and LeakyReLU activation. (7)； Where [·] indicates a channel splicing operation; Step 8: Generate a spatial attention weight map using the Sigmoid function: (8)； Where σ(·) represents the Sigmoid activation function; Step 9: In the attention calculation, a residual form is used to fuse the features after the spatial attention weights of the mask are dynamically adjusted with the features before adjustment: (9)； Where ⊙ represents element-wise multiplication, and F´ represents the output feature; Step 10: NIRST Image Reconstruction The fused features are upsampled step by step by the decoder, and skip connections corresponding to the encoder layer are introduced to fuse features, gradually restoring the resolution, and finally outputting the reconstructed result of the optical characteristic parameter distribution.

Images