Mama fluorescence tomography system based on pixel-region three-way scanning

Abstract

The invention relates to the technical field of optical molecular imaging, in particular to a fluorescent molecular tomography three-dimensional reconstruction system, method and device based on pixel and region integration layering Mama (PR3SM) and a three-dimensional scanning (Tri-scan) mechanism. Aiming at the problems of how to effectively capture global and local spatial characteristics of a fluorescence signal, how to accurately model a radial aggregation and diffusion mode of a fluorescence probe and how to realize a self-adaptive loss function to improve the reconstruction precision, the invention provides a layered three-way scanning Mama fluorescence molecular tomography reconstruction system based on pixel-region integration. The reconstruction precision is improved by introducing a Mama model, the feature modeling capability is enhanced by combining a layered PR-Mama module with a pixel-level SSM and a region-level SSM, forward, backward and spiral center scanning modes are fused through a three-way scanning mechanism, and a fluorophore distribution mode is effectively captured; the adaptive structure senses the loss function, integrates the mean square error and the local structure similarity loss, and improves the adaptability of the loss function.

Description

Technical Field

[0001] The present invention relates to the field of optical molecular imaging technology, and in particular to a fluorescence molecular tomography three-dimensional reconstruction system, method and device based on a pixel and region integrated hierarchical Mamba (PR3SM) and a three-dimensional scanning (Tri-scan) mechanism. Background Art

[0002] Fluorescence molecular tomography (FMT) is a highly sensitive optical molecular imaging technique widely used for early detection and treatment evaluation in small animal tumor models. However, FMT imaging suffers from low depth resolution and insufficient spatial resolution, primarily due to the strong scattering of photons within biological tissues and the ill-posed nature of the inverse problem. Traditional reconstruction methods, such as iterative algorithms based on the L1 / L2 norm and convolutional neural networks (CNNs), are limited by their local receptive fields and struggle to model long-range dependencies. Transformer-based methods, while capable of capturing global features, suffer from high computational complexity and struggle to process high-resolution data.

[0003] In recent years, the Mamba architecture within the state-space model (SSM) has demonstrated advantages in vision tasks due to its linear computational complexity and ability to model long sequences. However, existing technologies have yet to apply it to FMT reconstruction. Therefore, an efficient reconstruction system that combines local pixel-level features with global region-level features is urgently needed to improve the depth resolution and spatial localization accuracy of fluorescent targets. Summary of the Invention

[0004] 1. How to effectively capture the global and local spatial features of fluorescence signals: Existing methods are insufficient in capturing the global and local spatial features of fluorescence signals, especially when processing high-resolution images, which have high computational complexity and are prone to losing detail information.

[0005] 2. How to accurately model the radial aggregation and diffusion patterns of fluorescent probes: The distribution of fluorescent probes in biological tissues exhibits a radial aggregation and diffusion pattern. Existing methods have difficulty accurately capturing this characteristic, resulting in deviations in the reconstruction results in the structural and positional information of the target area.

[0006] 3. How to implement an adaptive loss function to improve reconstruction accuracy: Existing technologies lack a mechanism that can adaptively adjust the local structural similarity loss calculation area. This makes it difficult to accurately capture the structural information of the target region and dynamically balance the contribution of the loss function when the area of ​​the target region changes. This results in insufficient structural similarity and detail restoration in the reconstructed image.

[0007] In response to the above technical problems, the present invention provides a layered three-dimensional scanning Mamba fluorescence molecular tomography reconstruction system based on pixel-region integration.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] A hierarchical three-dimensional scanning Mamba fluorescence molecular tomography reconstruction system based on pixel-region integration, wherein the system adopts a 4-order multi-scale encoder-decoder U-shaped architecture;

[0010] The encoder inputs fluorescence data, generates a feature map through deep convolution, and processes it through the PR-3ScanMamba module to generate a multi-scale feature representation;

[0011] The decoder is responsible for recovering the encoded features, reconstructing the fluorophore distribution, and reducing the loss of spatial information through skip connections;

[0012] The PR-3ScanMamba module adopts a hierarchical design, consisting of a pixel-level SSM (PiM) submodule and a region-level SSM (ReM) submodule, and also incorporates a tri-scan mechanism.

[0013] The pixel-level SSM (PiM) is used to capture the local neighborhood pixel-level features of the fluorescence signal;

[0014] The regional-level SSM (ReM) is used to capture the long-range global patch-level features of the fluorescence signal;

[0015] Pixel-level SSM (PiM) and region-level SSM (ReM) work together to enhance the model's ability to learn detail representations at different scales, thereby improving the accuracy and detail restoration of reconstructed images.

[0016] The tri-scan mechanism includes forward scanning, backward scanning, and spiral center scanning. The spiral center scanning is designed based on the radial aggregation and diffusion characteristics of fluorophores to enhance the model's ability to model fluorophore distribution.

[0017] The fluorescence molecular tomography reconstruction method of the layered three-dimensional scanning Mamba fluorescence molecular tomography reconstruction system based on pixel-region integration includes the following steps:

[0018] Step 1: Fluorescence projection data acquisition: A charge coupled device (CCD) camera is used to collect real physical phantom experimental data as the data input of the system.

[0019] Step 2: Adaptive structure-aware loss function construction;

[0020] Step 3: Three-dimensional reconstruction of the fluorescent target. The PR3SM-FMT network architecture is used as the reconstruction model framework. Then, the collected data is input into the framework, and the optimal parameter model is saved through optimization learning. Finally, the saved optimal parameter model is used to perform system testing and three-dimensional reconstruction to obtain a three-dimensional spatial distribution image of the fluorescent target with depth resolution capability.

[0021] Furthermore, the adaptive structure-aware loss function includes a mean squared error (MSE) function and a local structural similarity loss function (Percep; Local Structural Similarity IndexMeasure, PSSIM).

[0022] Furthermore, the method for constructing the adaptive structure-aware loss function is: first, automatically adjust the calculation area of ​​the local structure similarity loss; then, ensure that the contributions of the mean square error function and the local structure similarity loss function are dynamically balanced in different reconstruction scenarios, thereby achieving a fine depiction of the details of the fluorescent target area and improving the quality and accuracy of the reconstructed image.

[0023] Furthermore, the step 3 specifically includes the following steps:

[0024] Step 3.1, data preprocessing: normalize the collected fluorescence projection data to eliminate the influence of data dimension and improve the stability of network training;

[0025] Step 3.2, network initialization: Randomly initialize the weight parameters of the PR3SM-FMT network and set reasonable training hyperparameters such as learning rate, batch size, etc.

[0026] Step 3.3, model training: input the preprocessed data into the network, calculate the network output through forward propagation, calculate the loss value using the adaptive structure-aware loss function, and then update the network parameters through the backpropagation algorithm to continuously optimize the network's reconstruction performance; during the training process, regularly save the network model and evaluate it on the validation set to prevent overfitting;

[0027] Step 3.4, model testing: Select a set of independent test data and use the trained PR3SM-FMT model to perform 3D reconstruction. Evaluate the model's reconstruction accuracy, spatial resolution, and contrast indicators to verify the model's performance and effectiveness.

[0028] Step 3.5, visualization of 3D reconstruction results: The reconstructed 3D spatial distribution image of the fluorescent target is visualized to more intuitively observe the position, size, and shape information of the fluorescent target, providing a basis for subsequent analysis and application.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] 1. Improve reconstruction accuracy: The Mamba model is introduced, which has the advantages of linear complexity, global receptive field, and dynamic weights. It can process long sequence data while retaining detailed information. It achieves global sensitivity through long-range dependencies and improves the accuracy of FMT reconstruction tasks.

[0031] 2. Enhanced feature modeling capabilities: The hierarchical PR-Mamba module combines pixel-level SSM and region-level SSM to enhance local neighborhood pixel-level and long-distance global patch-level feature modeling capabilities, improving the model's ability to learn detailed representations at different scales.

[0032] 3. Effectively capture the fluorophore distribution pattern: The three-way scanning mechanism integrates forward, backward and spiral center scanning modes to effectively capture the radial correlation, spatial aggregation and diffusion patterns of fluorophore distribution, improving the model's ability to model fluorophore distribution.

[0033] 4. Improve the adaptability of the loss function: The adaptive structure-aware loss function integrates the mean square error and local structural similarity loss, automatically adjusts the calculation area, and dynamically balances the contributions of the two to achieve fine depiction of the details of the fluorescent target area and improve the quality of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of the overall architecture of the system of the present invention, illustrating the complete process from fluorescence projection data acquisition to 3D reconstruction. Data acquisition includes physical phantom adjustment, fluorescence projection images, excitation light projection images, and white light projection data. After preprocessing, this data is input into the PR3SM-FMT network for reconstruction, ultimately obtaining a 3D spatial distribution image of the fluorescent target.

[0035] Figure 2 This is the encoder-decoder architecture diagram of the PR3SM-FMT network, showing that the encoder encodes the input data into a multi-scale feature map through deep convolution. After processing by the PR-3ScanMamba module, the features are passed to the decoder through jump connections to achieve gradual restoration of features and reconstruction of the fluorescent target distribution.

[0036] Figure 3 The PR-3ScanMamba module architecture diagram details the data processing workflow for pixel-level SSM (PiM) and region-level SSM (ReM). The PiM module models inter-pixel dependencies to capture local features, while the ReM module models global dependencies and integrates regional information. This layered design enhances the ability to capture both local and global features of fluorescence signals.

[0037] Figure 4A diagram of the three-dimensional scanning mechanism's path design, showing the forward, backward, and spiral center scanning paths. The spiral center scanning mechanism, designed to address the radial aggregation and diffusion characteristics of fluorophores, efficiently covers the data space, capturing the changing trends of fluorescence intensity and its correlation with surrounding pixels. DETAILED DESCRIPTION

[0038] To gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description thereof. However, the present invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a comprehensive understanding of the present disclosure.

[0039] The specific embodiments of the present invention are further explained below.

[0040] A hierarchical three-dimensional scanning Mamba fluorescence molecular tomography reconstruction system based on pixel-region integration, such as Figure 1 As shown, it mainly includes the following specific steps.

[0041] Step 1: Fluorescence projection data acquisition:

[0042] The specific process is as follows: First, the physical phantom to be imaged is aligned with the center of the charge-coupled device (CCD) camera's field of view. Next, a fluorescence projection image is acquired for 3D reconstruction, with a CCD exposure time of 0.2 seconds. Next, an excitation light projection image is acquired for Born normalization, with a CCD exposure time of 0.02 seconds. Finally, white light projection data is acquired for boundary contour extraction, with a CCD exposure time of 0.1 seconds. These data serve as input to the subsequent reconstruction system, providing foundational information for fluorescence molecular tomography reconstruction.

[0043] Step 2: PR3SM-FMT network architecture design:

[0044] The PR3SM-FMT adopts a four-layer multi-scale encoder-decoder U-shaped architecture. Figure 2 As shown, when the resolution is After the fluorescence data flows into the encoder, the depthwise convolution first encodes the input into a feature map Then the feature map The multi-scale feature maps are sent to each PR-3ScanMamba block to obtain a multi-scale feature map. A PR-3ScanMamba module contains two sub-modules: pixel-level SSM (PiM) and region-level SSM (ReM). For the first layer, the processing process can be expressed as:

[0045] (1)

[0046] In the subsequent stage, feature maps The PR-3ScanMamba module was processed in the same way as in the initial stage.

[0047] After each stage, the generated feature map Encoded as ,in Represents feature maps channels and resolution.

[0048] The decoder is responsible for gradually restoring these encoded features to the original fluorescent target distribution. Skip connections are set between the corresponding layers of the encoder and decoder to prevent the loss of spatial information during multiple sampling processes.

[0049] Step 3: PR-3ScanMamba Module Design:

[0050] The PR-3ScanMamba module is the core module of PR3SM-FMT. Figure 3 As shown in Figure 2, it utilizes a hierarchical design, primarily consisting of a pixel-level SSM (PiM) submodule and a region-level SSM (ReM) submodule, which are used for local and global modeling of fluorescence distribution characteristics, respectively. Furthermore, the module integrates three scanning modes: forward, backward, and spiral center scanning. The latter is specifically designed for the radially focused and diffuse structures of fluorescence distribution.

[0051] Pixel-level Mamba (PiM): The PiM submodule utilizes the Mamba layer to model complex inter-pixel dependencies. It can learn information such as fluorescence intensity trends between adjacent pixels and fluorescence distribution patterns within local regions. Based on factors such as the resolution and noise level of the acquired fluorescence data, as well as the size and distribution of the target, each image layer is divided into K×K non-overlapping subregions P. For an input feature map with a resolution of D×H×W, each H×W image layer is partitioned along the D direction into D×(H / K)×(W / K) subregions P. Based on this, the subregion size is dynamically adjusted based on the characteristics of the fluorescence data: when the fluorescence data is sparsely distributed and high resolution, the subregion size is reduced to capture fine features; when the distribution structure is simple, the subregion size is increased to improve computational efficiency.

[0052] Region-level Mamba (ReM): The ReM submodule is used to model the global dependencies among different sub-regions, ensuring that the model can comprehensively and accurately understand the overall characteristics of the fluorescence distribution data. Specifically, let the feature map output from the PiM submodule be denoted as X. First, X is passed through a k-size pooling layer (the step value is determined according to the data scale and the distinguishability of the features after downsampling), obtaining an aggregated map Y. Subsequently, to strengthen the information interaction between sub-regions, Mamba is used to further process the aggregated map Y. Finally, an inverse pooling operation is performed on the processed aggregated map to restore it to the same size as the initial feature map X, and a residual connection is used to effectively integrate the information of different sub-regions and learn the fluorescence correlation patterns between different regions. The entire process can be represented by the following formula:

[0053] (2)

[0054] Three-way scanning mechanism: The scanning mechanism is one of the cores of achieving high-performance reconstruction, and the three scanning methods are as shown in Figure 4 Forward scanning processes each element in the natural order of the input data, establishing a forward dependency relationship of the data. Backward scanning complements the information that forward scanning may have missed, especially when dealing with fluorescence distribution data with a circular structure, backward scanning can discover the reverse dependency relationship hidden in the data.

[0055] Spiral center scanning is specially designed for the special structure of fluorescence distribution. Since the distribution of fluorophores often presents a tendency of central aggregation and outward diffusion, spiral center scanning can sensitively identify this radial distribution characteristic. It starts from the central region of the data and expands outward in a spiral path. During the scanning process, it can efficiently cover the entire data space and comprehensively obtain the change trend of fluorescence intensity in different radial directions and its correlation with the surrounding pixels. PR3SM-FMT integrates these three scanning methods and proposes a Three-way scanning (TWS) Mamba block.

[0056] (3)

[0057] wherein the symbols f, b, and c represent flattening in the forward, backward, and spiral center directions, respectively.

[0058] Step 4: Design of an Adaptive Structure-Aware Loss Function: This adaptive structure-aware loss function comprehensively balances the mean squared error (MSE) and local structure similarity loss (PSSIM), prioritizing losses related to the target region. This loss function automatically adjusts the calculation region for the local structure similarity loss, ensuring that even in small target regions, the structural information of the target region is effectively captured, preventing it from being overlooked in the loss calculation. This adaptive adjustment mechanism dynamically balances the contributions of the mean squared error and local structure similarity loss based on different reconstruction scenarios, thereby achieving accurate depiction of the details of the fluorescent target region and improving the structural similarity and detail restoration of the reconstructed image in the target region.

[0059] . (4)

[0060] in, and It is a weight parameter used to adjust the relative importance, which can be adjusted and optimized according to the specific experimental conditions.

[0061] Step 5: 3D reconstruction of the fluorescent target. The real physical phantom experimental data collected in Step 1 is fed into the fluorescence molecular tomography reconstruction system constructed in Steps 2 and 3, which integrates pixel and region set hierarchical Mamba and three-dimensional scanning. The system can quickly perform 3D reconstruction, thereby obtaining a high-quality 3D spatial distribution image of the fluorescent target. The specific steps are as follows:

[0062] Data preprocessing: The collected fluorescence projection data is normalized to eliminate the influence of data dimension and improve the stability of network training.

[0063] Network initialization: Randomly initialize the weight parameters of the PR3SM-FMT network and set reasonable training hyperparameters such as learning rate and batch size.

[0064] Model training: Preprocessed data is fed into the network, and the network output is calculated through forward propagation. The loss value is calculated using an adaptive structure-aware loss function. The network parameters are then updated through a backpropagation algorithm to continuously optimize the network's reconstruction performance. During training, the network model is periodically saved and evaluated on a validation set to prevent overfitting.

[0065] Model testing: Select a set of independent test data and use the trained PR3SM-FMT model to perform 3D reconstruction. Evaluate the model's reconstruction accuracy, spatial resolution, contrast, and other indicators to verify the model's performance and effectiveness.

[0066] Visualization of 3D reconstruction results: The reconstructed 3D spatial distribution image of the fluorescent target is visualized to more intuitively observe the position, size, shape and other information of the fluorescent target, providing a basis for subsequent analysis and application.

[0067] Any matters not described in detail in this specification are prior art known to those skilled in the art. Although the above description of the present invention is based on specific embodiments to facilitate understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the specific embodiments. As long as various modifications are within the spirit and scope of the present invention as defined and determined by the appended claims, such modifications will be obvious to those skilled in the art, and all inventions and creations utilizing the concepts of the present invention are protected.

Claims

1. The Mamba fluorescence tomography system based on pixel-area three-dimensional scanning is characterized by: The system adopts a 4-order multi-scale encoder-decoder U-shaped architecture; The encoder inputs fluorescence data, generates a feature map through deep convolution, and processes it through the PR-3ScanMamba module to generate a multi-scale feature representation; The decoder is responsible for recovering the encoded features, reconstructing the fluorophore distribution, and reducing the loss of spatial information through skip connections; The PR-3ScanMamba module adopts a hierarchical design, consisting of a pixel-level SSM submodule and a region-level SSM submodule, and also integrates a three-way scanning mechanism; The pixel-level SSM is used to capture the local neighborhood pixel-level features of the fluorescence signal; The regional-level SSM is used to capture the long-range global patch-level features of the fluorescence signal; The three-directional scanning mechanism includes forward scanning, backward scanning and spiral center scanning.

2. The fluorescence molecular tomography reconstruction method of the Mamba fluorescence tomography imaging system based on pixel-area three-dimensional scanning according to claim 1, characterized in that: The following steps are involved: Step 1: Fluorescence projection data acquisition: using a charge-coupled device camera to collect real physical phantom experimental data as the data input of the system; Step 2: Adaptive structure-aware loss function construction; Step 3: Three-dimensional reconstruction of the fluorescent target. The PR3SM-FMT network architecture is used as the reconstruction model framework. Then, the collected data is input into the framework, and the optimal parameter model is saved through optimization learning. Finally, the saved optimal parameter model is used to perform system testing and three-dimensional reconstruction to obtain a three-dimensional spatial distribution image of the fluorescent target with depth resolution capability.

3. The fluorescence molecular tomography reconstruction method according to claim 2, characterized in that: The adaptive structure-aware loss function includes a mean square error function and a local structure similarity loss function.

4. The fluorescence molecular tomography reconstruction method according to claim 2, characterized in that: The method for constructing the adaptive structure-aware loss function is: first, automatically adjust the calculation area of ​​the local structure similarity loss; then, ensure that the contributions of the mean square error function and the local structure similarity loss function are dynamically balanced in different reconstruction scenarios, thereby achieving a fine depiction of the details of the fluorescent target area and improving the quality and accuracy of the reconstructed image.

5. The fluorescence molecular tomography reconstruction method according to claim 2, characterized in that: The step 3 specifically includes the following steps: Step 3.1, data preprocessing: normalize the collected fluorescence projection data; Step 3.2, network initialization: randomly initialize the weight parameters of the PR3SM-FMT network and set the training hyperparameters; Step 3.3, model training: input the preprocessed data into the network, calculate the network output through forward propagation, calculate the loss value using the adaptive structure-aware loss function, and then update the network parameters through the backpropagation algorithm to continuously optimize the network's reconstruction performance; during the training process, regularly save the network model and evaluate it on the validation set to prevent overfitting; Step 3.4, model testing: Select a set of independent test data and use the trained PR3SM-FMT model to perform 3D reconstruction. Evaluate the model's reconstruction accuracy, spatial resolution, and contrast indicators to verify the model's performance and effectiveness. Step 3.5, visualization of the three-dimensional reconstruction results: Visualize the reconstructed three-dimensional spatial distribution image of the fluorescent target.

Images