Image determination method, apparatus, and electronic device

Abstract

The application provides an image determination method and device and electronic equipment, which can be applied to the technical field of medical image processing. The method comprises the following steps: performing masking on the overlapping area of an anatomical image and a target body surface fluorescent distribution image to obtain a target mask image, wherein the target body surface fluorescent distribution image represents the fluorescent intensity distribution of fluorescent molecules overflowing the body surface of a target object in three-dimensional space; reconstructing the target body surface fluorescent distribution image by using a target reconstruction function to obtain an initial fluorescent molecule tomographic image, wherein the initial fluorescent molecule tomographic image represents the initial density spatial distribution of the fluorescent molecules in tumor tissue; extracting structure prior features corresponding to the target mask image, density spatial features corresponding to the initial fluorescent molecule tomographic image, and optical features corresponding to the target body surface fluorescent distribution image; and fusing the structure prior features, the density spatial features and the optical features based on an attention mechanism to obtain a target fluorescent molecule tomographic image.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically to an image determination method, apparatus, and electronic device. Background Technology

[0002] Fluorescence molecular tomography (FMT) is a three-dimensional optical imaging technique based on diffusion optics. It reconstructs the three-dimensional distribution of bioluminescent activity within biological tissues to achieve tumor localization and volume estimation. However, due to the complex structure and high scattering characteristics of biological tissues, forward modeling in FMT inevitably contains errors, and the inverse problem exhibits a high degree of ill-conditioning. These factors collectively lead to increased artifacts and limited accuracy in reconstructed images. Therefore, an image reconstruction strategy is urgently needed to improve the quality of FMT images. Summary of the Invention

[0003] In view of the above problems, the present invention provides an image determination method, apparatus and electronic device.

[0004] According to a first aspect of the present invention, an image determination method is provided, comprising: masking overlapping regions of an anatomical image and a target body surface fluorescence distribution image to obtain a target mask image for marking tumor regions of a target object, wherein the target body surface fluorescence distribution image characterizes the fluorescence intensity distribution of fluorescent molecules overflowing from the target object's body surface in three-dimensional space, and the anatomical image characterizes the tissue structure of the target object; reconstructing the target body surface fluorescence distribution image using a target reconstruction function to obtain an initial fluorescent molecule tomographic image, wherein the initial fluorescent molecule tomographic image characterizes the initial density spatial distribution of fluorescent molecules in the tissue; extracting structural prior features corresponding to the target mask image, density spatial features corresponding to the initial fluorescent molecule tomographic image, and optical features corresponding to the target body surface fluorescence distribution image; and fusing the structural prior features, density spatial features, and optical features based on an attention mechanism to obtain a target fluorescent molecule tomographic image.

[0005] Optionally, the overlapping region of the anatomical image and the surface fluorescence distribution image is masked to obtain a target mask image, including: pixel registration of the anatomical image and the initial surface fluorescence distribution image to obtain a registration image, wherein the initial surface fluorescence distribution image represents the fluorescence intensity distribution of fluorescent molecules overflowing from the surface of the target object in two-dimensional space; meshing the registration image to obtain a three-dimensional mesh, wherein the three-dimensional mesh represents the three-dimensional morphology of the tissue structure of the target object; mapping the initial surface fluorescence distribution image to the three-dimensional mesh to obtain the target surface fluorescence distribution image; and masking the overlapping region of the three-dimensional mesh and the target surface fluorescence distribution image to obtain the target mask image.

[0006] Optionally, the overlapping region of the three-dimensional mesh and the fluorescence distribution image of the target body surface is masked to obtain a target mask image, including: processing the fluorescence distribution image of the target body surface using an adaptive threshold algorithm to obtain a target region, wherein the target region is the projection of the target fluorescence signal region in the fluorescence distribution image of the target body surface onto the three-dimensional space, and the intensity of the target fluorescence signal is greater than a preset threshold; and masking the region in the three-dimensional mesh that overlaps with the target region to obtain a target mask image.

[0007] Optionally, the registered image is meshed to obtain a three-dimensional mesh, including: segmenting the registered image using a multimodal segmentation algorithm to obtain an initial mask image; and meshing the initial mask image to obtain a three-dimensional mesh.

[0008] Optionally, the target surface fluorescence distribution image is initially reconstructed using a target reconstruction function to obtain an initial fluorescence molecular tomographic image. This includes: iteratively reconstructing the target surface fluorescence distribution image based on a conjugate gradient algorithm and target constraints to obtain the initial fluorescence molecular tomographic image. The conjugate gradient algorithm is used to optimize the fluorescence intensity distribution of the fluorescence molecular tomographic image reconstructed from the target surface fluorescence distribution image in the tumor region. The target constraints are used to constrain the fluorescence intensity distribution of the predicted surface fluorescence distribution image to be close to the fluorescence intensity distribution of the target surface fluorescence distribution image. The predicted surface fluorescence distribution image is obtained by transforming the reconstructed fluorescence molecular tomographic image based on a weight matrix. The weight matrix represents the mapping relationship between the surface fluorescence distribution image and the fluorescence molecular tomographic image constructed based on an optical transmission model.

[0009] Optionally, the target constraints may include at least a sparse regularization penalty term and a smoothing penalty term. The sparse regularization penalty term is used to constrain the fluorescence intensity distribution of the reconstructed fluorescent molecular tomographic image in the non-tumor region to meet the sparsity condition, and the smoothing penalty term is used to constrain the smoothness of the fluorescence intensity distribution of the reconstructed fluorescent molecular tomographic image in the tumor region to be greater than a preset smoothness threshold.

[0010] Optionally, the tomographic image of the target fluorescent molecule obtained by fusing structural prior features, density spatial features, and optical features based on an attention mechanism includes: multi-scale fusion of structural prior features, density spatial features, and optical features to obtain multi-scale features; structural correction of the multi-scale features based on the structural prior features to obtain structurally corrected features; spatial correction of the structurally corrected features based on the density spatial features to obtain spatially corrected features; and processing the spatially corrected features using a multi-head attention mechanism to obtain the tomographic image of the target fluorescent molecule.

[0011] Optionally, a multi-head attention mechanism is used to process the spatial correction features to obtain a tomographic image of the target fluorescent molecule. This includes: using a multi-head attention mechanism to fuse the value features, bond features, and query features determined based on the spatial correction features to obtain attention weights; using the attention weights to modulate the spatial correction features to obtain modulated features; extracting deconvolution features from the modulated features to obtain deconvolution features; performing fully connected processing on the deconvolution features to obtain fully connected features; and mapping the fully connected features back to three-dimensional space to obtain a tomographic image of the target fluorescent molecule.

[0012] A second aspect of the present invention provides an image determination apparatus, comprising: a masking module for masking overlapping regions of an anatomical image and a target surface fluorescence distribution image to obtain a target mask image for marking tumor regions of a target object, wherein the target surface fluorescence distribution image characterizes the fluorescence intensity distribution of fluorescent molecules overflowing from the target object's surface in three-dimensional space, and the anatomical image characterizes the internal tissue structure of the target object; an optimization module for reconstructing the target surface fluorescence distribution image using a target reconstruction function to obtain an initial fluorescence molecule tomographic image, wherein the initial fluorescence molecule tomographic image characterizes the initial density spatial distribution of fluorescent molecules in the tissue; a feature extraction module for extracting structural prior features corresponding to the target mask image, density spatial features corresponding to the initial fluorescence molecule tomographic image, and optical features corresponding to the target surface fluorescence distribution image; and a fusion module for fusing the structural prior features, density spatial features, and optical features based on an attention mechanism to obtain a target fluorescence molecule tomographic image.

[0013] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the method described above.

[0014] According to the image determination method, apparatus, and electronic device provided by the present invention, since the fluorescence distribution image of the target surface is first reconstructed based on the target reconstruction function, a high-quality initial fluorescence molecular tomographic image is obtained. This initial fluorescence molecular tomographic image can be used as a density space feature input to the neural network, providing structured probabilistic guidance for highly ill-conditioned reconstruction problems. This effectively reduces the solution space, stabilizes the iterative optimization process, significantly suppresses diffusion artifacts, improves the specificity and accuracy of target localization, and enhances the interpretability of the network. The optimal initial solution of the optimization process, using the initial fluorescence molecular tomographic image as the starting point, initiates a refined solution for global scattering artifact correction. This significantly shortens the iteration process and convergence time during optimization while improving the reconstruction accuracy of the target fluorescence molecular tomographic image. Furthermore... By integrating attention mechanisms with structural prior features, density spatial features, and optical features, multi-dimensional optimization and improvement were achieved. In terms of spatial accuracy, the use of structural prior features and attention weighting eliminated boundary and depth positioning deviations caused by scattering artifacts, providing guidance information for anatomical boundaries with spatial constraints. This limited the computational area during optimization and reduced computational load. In terms of artifact suppression, density spatial features represent probabilistic prior knowledge for fluorophore optimization, guiding attention to focus on high-probability regions. The optimization process prevents the blind solving of fluorescence molecular tomography images, effectively reducing false positives in fluorescence signals caused by ill-defined problems. As a result, the target fluorescence molecular tomography images possess both optical molecular specificity and anatomical structural fidelity, providing a more reliable basis for precise clinical diagnosis and surgical navigation. Attached Figure Description

[0015] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings.

[0016] Figure 1 A flowchart of an image determination method according to an embodiment of the present invention is shown.

[0017] Figure 2 A schematic diagram of the fluorescence distribution image on the surface of a target body according to an embodiment of the present invention is shown.

[0018] Figure 3 A schematic diagram of a target mask image according to an embodiment of the present invention is shown.

[0019] Figure 4 A schematic diagram of a target fluorescent molecular tomographic image according to an embodiment of the present invention is shown.

[0020] Figure 5 A schematic diagram of generating a target fluorescent molecular tomographic image according to an embodiment of the present invention is shown.

[0021] Figure 6 A structural block diagram of an image determination apparatus according to an embodiment of the present invention is shown.

[0022] Figure 7 A block diagram of an electronic device suitable for implementing an image determination method according to an embodiment of the present invention is shown. Detailed Implementation

[0023] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0027] In the process of realizing this invention, it was discovered that in related model-based reconstructions, the number of unknown variables (fluorescence source distribution) far exceeds the number of known variables (captured fluorescence signals), which is a typical ill-conditioned problem. Moreover, light transmission in biological tissues is mainly a scattering effect; therefore, the reconstructed fluorescence molecular tomography images suffer from numerous artifacts and poor imaging quality. Related deep learning methods are often based on fixed grids, which have low transferability and high computational cost.

[0028] In view of this, embodiments of the present invention provide an image determination method, apparatus, and electronic device. The method includes: masking the overlapping region of an anatomical image and a target surface fluorescence distribution image to obtain a target mask image, wherein the target surface fluorescence distribution image characterizes the fluorescence intensity distribution of fluorescent molecules overflowing from the target object's surface in three-dimensional space; reconstructing the target surface fluorescence distribution image using a target reconstruction function to obtain an initial fluorescent molecule tomographic image, wherein the initial fluorescent molecule tomographic image characterizes the initial density spatial distribution of fluorescent molecules in tumor tissue; extracting structural prior features corresponding to the target mask image, density spatial features corresponding to the initial fluorescent molecule tomographic image, and optical features corresponding to the target surface fluorescence distribution image; and fusing the structural prior features, density spatial features, and optical features based on an attention mechanism to obtain a target fluorescent molecule tomographic image.

[0029] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0030] It should be noted that the sequence numbers of the operations in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0031] Figure 1 A flowchart of an image determination method according to an embodiment of the present invention is shown.

[0032] like Figure 1 As shown, the method includes operations S110 to S140.

[0033] In operation S110, the overlapping area of ​​the anatomical image and the fluorescence distribution image of the target body surface is masked to obtain a target mask image for marking the tumor region of the target object.

[0034] In operation S120, the target reconstruction function is used to reconstruct the fluorescence distribution image on the target surface to obtain the initial fluorescence molecular tomographic image.

[0035] In operation S130, the structural prior features corresponding to the target mask image, the initial spatial density features corresponding to the initial fluorescence molecular tomography image, and the optical features corresponding to the fluorescence distribution image on the target surface are extracted.

[0036] In operation S140, prior structural features, initial spatial density features, and optical features are fused based on the attention mechanism to obtain tomographic images of the target fluorescent molecules.

[0037] Imaging equipment is used to scan the target object to obtain anatomical images, which characterize the internal tissue structure of the target object. Anatomical images can include computed tomography (CT) images, magnetic resonance imaging (MRI) images, ultrasound (US) images, or digital radiography (DR) images, etc.

[0038] The target object can be a tumor, such as a mouse with a tumor implanted.

[0039] When a specific wavelength of excitation light is irradiated onto the surface of a target object, fluorescent probes or proteins that accumulate in target tissues such as tumors absorb the excitation light and transition to a high-energy state. When they fall back, they emit fluorescence with a longer wavelength. This fluorescence is separated from the excitation light by a filter system and then collected by a high-sensitivity camera to finally form an image of the fluorescence distribution on the target surface, reflecting the spatial intensity distribution of the fluorescence signal on the surface.

[0040] The fluorescence distribution image on the target body surface represents the fluorescence intensity distribution of fluorescent molecules overflowing from the target body surface in three-dimensional space.

[0041] Figure 2 A schematic diagram of the fluorescence distribution image on the surface of a target body according to an embodiment of the present invention is shown.

[0042] like Figure 2 As shown, the coordinate axes X, Y, and Z represent three-dimensional space. The light yellow area represents the three-dimensional shape of the target object, and the other color brightness areas 210 represent the fluorescence intensity corresponding to the spatial position of the fluorescent molecules on the surface of the target object.

[0043] Anatomical images can reveal the location of every tissue detail, while areas of high fluorescence intensity in the target body surface fluorescence distribution image can indicate the location of the tumor. The anatomical image and the target body surface fluorescence distribution image are registered, and overlapping areas are masked to obtain a target mask image used to mark the tumor region of the target object.

[0044] Target mask images can accurately reflect the tissue in which the tumor is located and pinpoint the tumor region.

[0045] The fluorescence distribution image on the target surface is reconstructed using the target reconstruction function to obtain the initial fluorescence molecular tomographic image.

[0046] The target reconstruction function can be a regularization function based on the weight matrix. The target reconstruction function is used to perform initial iterative reconstruction of the fluorescence distribution image on the target surface to obtain an initial fluorescence molecular tomographic image.

[0047] Initial fluorescent molecular tomographic images characterize the initial spatial distribution of fluorescent molecules in tumor tissue. These images represent the three-dimensional spatial distribution of the fluorescent probe within the target object, reflecting features such as the location and size of the tumor tissue. However, due to optical scattering and absorption, the images exhibit diffusion artifacts and positional biases, requiring optimization.

[0048] Multi-scale feature extraction was performed on the initial fluorescent molecular tomographic image using a 3D U-shaped network (3D U-Net) to obtain initial features, which were then multi-scale fused features.

[0049] Feature extraction is performed on the target mask image using a convolutional encoder to obtain structural prior features. These structural prior features provide anatomical boundary guidance information for spatial constraints, which is used to limit the computational region during optimization, reducing computational cost and time.

[0050] The initial fluorescence molecular tomography image is subjected to feature extraction by a convolutional encoder to obtain density space features. The density space features are probabilistic prior knowledge that guides the optimization of fluorescence molecule tomography images. They are used to prevent the optimization of fluorescence molecular tomography images from blindly solving the problem, and instead find the optimal solution within a reasonable range of locations, thereby improving accuracy, reliability and network interpretability, and refining the computational range.

[0051] The fluorescence distribution on the target surface is extracted using a convolutional encoder to obtain optical features. These optical features, including optical intensity, local texture, and edge contours, serve as the underlying feature guide information and form the basis for the network's fluorescence molecular tomography reconstruction. These features directly encode the original photon distribution information received by the detector, exhibiting high fidelity and unambiguity. They provide the most direct and unabstracted data foundation for reconstruction, and their core function is to stabilize the reconstruction process and prevent bias caused by over-reliance on priors. On the other hand, by fusing with higher-level prior features, the reconstruction results can be constrained to remain consistent with physical observation facts, ensuring that the generated image maintains consistency with the original signal in detail, thereby improving the reliability and robustness of the reconstruction.

[0052] By utilizing an attention mechanism to fuse structural prior features, density spatial features, and optical features, information complementarity between different features is achieved. This breaks the static nature of serial or additive fusion, allowing the network to selectively focus on local features that are most relevant to structural prior information and most critical for reconstruction in both spatial and channel dimensions. At the same time, irrelevant or noise interference is suppressed, significantly improving the efficiency and accuracy of feature utilization. As a result, the final reconstructed target fluorescent molecule tomographic image not only fits the anatomical prior more structurally but also closely resembles the original optical signal in detail, demonstrating superior comprehensive performance in localization and artifact suppression.

[0053] Optionally, since a high-quality initial fluorescence molecular tomographic image is obtained by first reconstructing the fluorescence distribution image of the target surface based on the target reconstruction function, this initial fluorescence molecular tomographic image can be used as a density space feature input to the neural network. This provides structured probabilistic guidance for highly ill-conditioned reconstruction problems, effectively reducing the solution space, stabilizing the iterative optimization process, significantly suppressing diffusion artifacts, improving the specificity and accuracy of target localization, and enhancing the interpretability of the network. The optimal initial solution of the optimization process is used as the starting point to initiate a refined solution for global scattering artifact correction. This significantly shortens the iteration process and convergence time in the optimization process while improving the reconstruction accuracy of the target fluorescence molecular tomographic image. Furthermore, the attention mechanism is used to fuse and... By incorporating structural prior features, density spatial features, and optical features, multi-dimensional optimization and improvement have been achieved. In terms of spatial accuracy, the use of structural prior features and attention weighting eliminates boundary and depth positioning deviations caused by scattering artifacts, providing guidance information for anatomical boundaries with spatial constraints. This limits the computational area and reduces computational load during optimization. Regarding artifact suppression, density spatial features represent probabilistic prior knowledge for fluorophore optimization, guiding attention to focus on high-probability regions. The optimization process ensures that fluorescence molecular tomography image optimization is no longer blindly solved, effectively reducing false positives in fluorescence signals caused by ill-defined problems. As a result, the target fluorescence molecular tomography image possesses both optical molecular specificity and anatomical structural fidelity, providing a more reliable basis for precise clinical diagnosis and surgical navigation.

[0054] Optionally, the overlapping region of the anatomical image and the surface fluorescence distribution image is masked to obtain a target mask image, including: pixel registration of the anatomical image and the initial surface fluorescence distribution image to obtain a registration image, wherein the initial surface fluorescence distribution image represents the fluorescence intensity distribution of fluorescent molecules overflowing from the surface of the target object in two-dimensional space; meshing the registration image to obtain a three-dimensional mesh, wherein the three-dimensional mesh represents the three-dimensional morphology of the tissue structure of the target object; mapping the initial surface fluorescence distribution image to the three-dimensional mesh to obtain the target surface fluorescence distribution image; and masking the overlapping region of the three-dimensional mesh and the target surface fluorescence distribution image to obtain the target mask image.

[0055] An initial surface fluorescence distribution image was obtained by acquiring images of the target object's surface using a fluorescence imaging device. This initial surface fluorescence distribution image characterizes the fluorescence intensity distribution of fluorescent molecules overflowing from the target object's surface in two-dimensional space.

[0056] The initial surface fluorescence distribution image is raw data that has not been spatially registered with anatomical images. The acquired fluorescence signal is affected by scattering and absorption by the internal tissue of the target object because it has passed through biological tissue, and the acquired two-dimensional image cannot be directly combined with the three-dimensional structure.

[0057] By optimizing the spatial transformation parameters, pixel-level spatial alignment between the anatomical image and the initial surface fluorescence distribution image is achieved, thereby accurately superimposing the fluorescence signal onto the anatomical structure to obtain a registered and fused image. The fluorescence intensity distribution in the registered and fused image corresponds one-to-one with the anatomical structure in space, realizing a unified mapping between function and structure.

[0058] The spatial location and attributes (such as fluorescence intensity and texture information) of each pixel in the registered image can be mapped to the corresponding vertices or faces of the mesh, thus forming a three-dimensional mesh that carries multimodal information and can be used for quantitative analysis and interactive visualization. The three-dimensional mesh is a three-dimensional data structure.

[0059] The three-dimensional mesh is a geometric model of the tumor boundary of the target object, consisting of vertices and triangular facets.

[0060] The intrinsic parameters (focal length, principal point) and extrinsic parameters (position and orientation relative to the 3D mesh coordinate system) of the fluorescence imaging device are obtained through camera calibration, and a projection model from 3D world coordinates to 2D image pixel coordinates is established. Then, for a surface vertex on the 3D mesh, the corresponding pixel position on the initial surface fluorescence distribution image is calculated using this projection model. Fluorescence intensity values ​​of the vertex are obtained by sampling from neighboring pixels around the vertex using bilinear interpolation and assigned to the vertex, resulting in the target surface fluorescence distribution image. This image is a fluorescence intensity field with spatial coordinates attached to the 3D mesh, with each vertex assigned a fluorescence intensity value.

[0061] The overlapping region is a region in three-dimensional space that simultaneously meets the following two conditions. In terms of geometric conditions: the overlapping region belongs to a part of the three-dimensional mesh; in terms of functional conditions: the fluorescence intensity value attached to the mesh of the overlapping region exceeds a preset threshold, that is, it is a tumor anatomical surface region that expresses a fluorescence signal.

[0062] Mask the overlapping areas, marking the overlapping areas as 1 and the non-overlapping areas as 0, to obtain the target mask image.

[0063] The target mask image is a data structure isomorphic to a 3D mesh, but each element stores a binary label that marks the precise spatial extent of the "fluorescent positive region" on the 3D anatomical surface.

[0064] Optionally, the initial surface fluorescence distribution image is mapped onto a three-dimensional mesh, and then the overlapping area of ​​the three-dimensional mesh and the target surface fluorescence distribution image is masked. This corrects the signal distortion caused by the viewing angle and surface curvature in the initial surface fluorescence distribution image, so that the fluorescence signal can be presented realistically and quantitatively on the three-dimensional anatomical surface, and the fluorescence signal is accurately combined with the shape of the tumor boundary.

[0065] Optionally, the overlapping region of the three-dimensional mesh and the fluorescence distribution image of the target body surface is masked to obtain a target mask image, including: processing the fluorescence distribution image of the target body surface using an adaptive threshold algorithm to obtain a target region, wherein the target region is the projection of the target fluorescence signal region in the fluorescence distribution image of the target body surface onto the three-dimensional space, and the intensity of the target fluorescence signal is greater than a preset threshold; and masking the region in the three-dimensional mesh that overlaps with the target region to obtain a target mask image.

[0066] Imaging systems all have inherent noise. The preset threshold is usually set based on the statistical value of the signal intensity in the background area (such as non-fluorescent tissue far from the target). A signal intensity greater than the preset threshold means that the signal intensity exceeds the fluctuation range of random noise with high confidence, and is a valid positive signal.

[0067] Fluorescent signals are typically generated by specific probes (such as fluorescent markers targeting tumors, inflammation, or specific proteins). When the intensity of the target fluorescent signal is greater than a preset threshold, it indicates the presence of a sufficient concentration of the probe at that location, thereby indirectly indicating that the targeted biological process is occurring (such as tumor growth, infection, or gene expression).

[0068] An adaptive thresholding algorithm is used to process the fluorescence signal intensity of pixels in the fluorescence distribution image of the target body surface, and pixels with fluorescence signal intensity greater than a preset threshold are located to obtain the target region.

[0069] Mask the regions in the 3D mesh that overlap with the target region, marking the overlapping regions as 1 and the non-overlapping regions as 0 to obtain the target mask image.

[0070] Figure 3 A schematic diagram of a target mask image according to an embodiment of the present invention is shown.

[0071] like Figure 3 As shown, the coordinate axes X, Y, and Z represent three-dimensional space. The light yellow area represents the three-dimensional shape of the target object, and the purple area 310 represents the region in the three-dimensional mesh that overlaps with the target area.

[0072] Optionally, an adaptive thresholding algorithm is used to extract high fluorescence signal regions and remove background regions. Considering the high scattering characteristics of tissues, the high fluorescence signal regions are extended to obtain the target region. The target region is calculated and projected along the signal acquisition direction to form the spatial extension region. All voxels that intersect with the anatomical structure within the overlapping region in the three-dimensional grid are found to form the target mask image. This method can accurately separate and extract non-specific background noise from effective positive signals in space, providing a reliable and accurate geometric definition basis for subsequent extraction of prior features.

[0073] Optionally, the registered image is meshed to obtain a three-dimensional mesh, including: segmenting the registered image using a multimodal segmentation algorithm to obtain an initial mask image; and meshing the initial mask image to obtain a three-dimensional mesh.

[0074] Multimodal segmentation algorithms can be constructed based on U-shaped convolutional networks (U-Net).

[0075] A multimodal segmentation algorithm is used to segment the registered image to obtain an initial mask image. The initial mask image is a binary image, where a pixel value of 1 represents a tissue pixel of the target object, and a pixel value of 0 represents a non-tissue pixel of the target object.

[0076] The initial mask image can be meshed using isosurface extraction or surface reconstruction algorithms to calculate continuous geometric surfaces from the discrete three-dimensional voxel mask, thus obtaining a three-dimensional mesh.

[0077] Optionally, the target surface fluorescence distribution image is initially reconstructed using a target reconstruction function to obtain an initial fluorescence molecular tomographic image. This includes: iteratively reconstructing the target surface fluorescence distribution image based on a conjugate gradient algorithm and target constraints to obtain the initial fluorescence molecular tomographic image. The conjugate gradient algorithm is used to optimize the fluorescence intensity distribution of the fluorescence molecular tomographic image reconstructed from the target surface fluorescence distribution image in the tumor region. The target constraints are used to constrain the fluorescence intensity distribution of the predicted surface fluorescence distribution image to be close to the fluorescence intensity distribution of the target surface fluorescence distribution image. The predicted surface fluorescence distribution image is obtained by transforming the reconstructed fluorescence molecular tomographic image based on a weight matrix. The weight matrix represents the mapping relationship between the surface fluorescence distribution image and the fluorescence molecular tomographic image constructed based on an optical transmission model.

[0078] In the initial reconstruction of the fluorescence distribution image of the target body surface using the target reconstruction function, the conjugate gradient algorithm is used to solve for and optimize the initial fluorescence molecular tomography image.

[0079] In the conjugate gradient algorithm, the maximum number of iterations is set to 100, the line search parameter α is set to 0.01, and the scalar coefficient β is set to 0.6.

[0080] In one embodiment, the target reconstruction function L is as shown in formula (1):

[0081] (1).

[0082] in, Let X be the L1 norm. Let X be the total variation. , Here are two regularization weights, W is the weight matrix, and X is the fluorescence molecular tomography image obtained during the iterative reconstruction process. Image showing the fluorescence distribution on the target surface.

[0083] The optical transport model can be an approximation of the radiative transport model, and the weight matrix is ​​used to describe the conversion relationship between the fluorescence distribution image of the target surface and the fluorescence molecular tomography image.

[0084] The fluorescence intensity distribution of the predicted surface fluorescence distribution image is based on the weight matrix. Fluorescent molecular tomography images obtained during the iterative reconstruction process The result is obtained through conversion.

[0085] The objective of the constraint condition is to minimize the fluorescence intensity distribution of the predicted surface fluorescence distribution image. Fluorescence intensity distribution compared to the fluorescence distribution image of the target body surface The differences between them.

[0086] Optionally, the complex inverse problem of retrieving internal light sources from surface signal intensity is transformed into a constrained optimization problem of the target, and the conjugate gradient algorithm is used to solve it efficiently. This ensures that the reconstructed internal fluorescence source (tumor region) not only satisfies the physical model of light propagation in biological tissue, but also that the surface fluorescence signal mapped by the initial fluorescence molecular tomography image is highly consistent with the actual measured fluorescence distribution on the target surface. This effectively overcomes the pathological nature of the reconstruction problem, significantly suppresses the non-uniqueness of the solution and the influence of noise amplification, and thus obtains a more accurate and reliable three-dimensional distribution of tumor fluorescence in anatomy.

[0087] Optionally, the target constraints may include at least a sparse regularization penalty term and a smoothing penalty term. The sparse regularization penalty term is used to constrain the fluorescence intensity distribution of the reconstructed fluorescent molecular tomographic image in the non-tumor region to meet the sparsity condition, and the smoothing penalty term is used to constrain the smoothness of the fluorescence intensity distribution of the reconstructed fluorescent molecular tomographic image in the tumor region to be greater than a preset smoothness threshold.

[0088] The constraints of the objective constraints also include smoothness constraints and sparsity regularization penalties.

[0089] The smoothing penalty term is a regularization term used to constrain the smoothness of the initial fluorescent molecular tomographic image obtained during the optimization process to be greater than a preset smoothness threshold.

[0090] The sparsity condition is measured and penalized by the L1 norm (the sum of the absolute values ​​of fluorescence intensity of all voxels). Minimizing the L1 norm by the sparsity regularization penalty term will cause the fluorescence intensity of a large number of voxels to be pushed toward zero, thereby achieving sparsity of the overall distribution.

[0091] The sparse regularization penalty term is calculated for the fluorescence intensity values ​​of non-tumor regions in the reconstructed fluorescence molecular tomography image, yielding the L1 norm, which is then multiplied by a regularization parameter. To reduce the value of the sparsity regularization penalty term, the conjugate gradient algorithm is constrained to actively compress the fluorescence intensity of non-tumor regions lacking data support to zero or a very small value. This results in the final initial fluorescence molecular tomography image automatically satisfying the sparsity condition in spatial distribution, meaning that fluorescence is concentrated in only a few prominent clusters, while the background area is clean and clear.

[0092] Optionally, the smoothness of the fluorescent molecular tomographic image in the target constraint iteration is greater than a preset smoothness threshold, thereby ensuring that the initial fluorescent molecular tomographic image obtained by the final optimization has a certain smoothness and projectability.

[0093] Optionally, the tomographic image of the target fluorescent molecule obtained by fusing structural prior features, density spatial features, and optical features based on an attention mechanism includes: multi-scale fusion of structural prior features, density spatial features, and optical features to obtain multi-scale features; structural correction of the multi-scale features based on the structural prior features to obtain structurally corrected features; spatial correction of the structurally corrected features based on the density spatial features to obtain spatially corrected features; and processing the spatially corrected features using a multi-head attention mechanism to obtain the tomographic image of the target fluorescent molecule.

[0094] A training set was obtained, consisting of sample images and labels for 112 target objects. The sample images included target mask images, initial fluorescence molecular tomographic images, and surface fluorescence distribution images of the target objects. Prior structural features of the target objects corresponding to the target mask images, spatial density features of the target objects corresponding to the initial fluorescence molecular tomographic images, and optical features of the target objects corresponding to the surface fluorescence distribution images were extracted.

[0095] The sample density spatial features, sample structure prior features, and sample optical features are input into a Transformer encoding network and a multi-scale feature decoding network based on a multi-head attention mechanism, and the output is a sample target fluorescent molecular tomographic image.

[0096] The Dice similarity coefficient or cross-entropy loss function can be combined to calculate the loss value between the sample target fluorescent molecular tomographic image and the sample label to train the network for parameter tuning. Adjustable weight parameters can be set to balance region overlap and pixel classification accuracy. The network parameters can be optimized through gradient backpropagation. A cosine annealing strategy is used to periodically adjust the learning rate, and the weight decay mechanism of the optimizer (Adam with Decoupled Weight Decay, AdamW) is used to achieve efficient convergence and stable training of the model.

[0097] By fusing initial features, structural prior features, and position prior features using a trained model, a tomographic image of the target fluorescent molecule is obtained.

[0098] Channel splicing is performed on structural prior features, density space features, and optical features, and multi-scale features are obtained through convolution dimensionality reduction and enhancement.

[0099] Structural correction features are obtained by taking the dot product of structural prior features and multi-scale features. These structural correction features are the local features most relevant to the structural prior information in the anatomical structure and most critical for reconstruction.

[0100] The density spatial features and structural correction features are spliced ​​together to obtain the spatial correction features. The spatial correction features are soft-constrained features of the structural correction features, and the spatial correction features are closer to the original fluorophore signal in terms of signal information details.

[0101] Multi-head attention mechanism is used to fuse spatially corrected features to obtain tomographic images of target fluorescent molecules.

[0102] Figure 4 A schematic diagram of a target fluorescent molecular tomographic image according to an embodiment of the present invention is shown.

[0103] like Figure 4 As shown, the coordinate axes X, Y, and Z represent three-dimensional spatial information. The light yellow area represents the three-dimensional shape of the target object, and the orange area 410 represents the actual location and boundary of the tumor reconstructed in the target fluorescent molecular tomography image.

[0104] Optionally, based on prior structural features and density spatial features, the model is guided to dynamically and selectively focus on potential fluorescent source regions in optical features that are highly consistent with prior knowledge and have reasonable spatial locations. This effectively suppresses reconstruction artifacts caused by photon scattering and noise, improves the coherence of tumor boundaries, and enhances the spatial accuracy, signal specificity, and biological reliability of the target image. In addition, by using the initial fluorescent molecular tomographic image as the location and combining multiple prior features for optimized reconstruction, the reconstruction efficiency is improved.

[0105] Optionally, a multi-head attention mechanism is used to process the spatial correction features to obtain a tomographic image of the target fluorescent molecule. This includes: using a multi-head attention mechanism to fuse the value features, bond features, and query features determined based on the spatial correction features to obtain attention weights; using the attention weights to modulate the spatial correction features to obtain modulated features; extracting deconvolution features from the modulated features to obtain deconvolution features; performing fully connected processing on the deconvolution features to obtain fully connected features; and mapping the fully connected features back to three-dimensional space to obtain a tomographic image of the target fluorescent molecule.

[0106] Convolution processing is performed on the spatially corrected features to obtain value features, key features, and query features.

[0107] The global context relationship is modeled based on a multi-head attention mechanism. Attention is fused between value features, key features, and query features. Then, the feature representation capability is enhanced by multilayer perceptron extension and residual connection to obtain attention weights.

[0108] The attention weights are fused with the spatial correction features, and the fused features are then residually integrated with the density spatial features to obtain the modulation features.

[0109] The modulation features are extracted by using a deconvolution module to obtain deconvolution features. These deconvolution features are used to generate decoding information for fine prediction by gradually restoring spatial resolution.

[0110] The structural prior features are upsampled to the output size, and the upsampled features and deconvolution features are concatenated and fully connected to obtain fully connected features.

[0111] By mapping the fully connected features back to three-dimensional space, a tomographic image of the target fluorescent molecule is obtained.

[0112] Figure 5 A schematic diagram of generating a target fluorescent molecular tomographic image according to an embodiment of the present invention is shown.

[0113] like Figure 5 As shown, pixel registration is performed on the anatomical image and the initial surface fluorescence distribution image to obtain a registered image; the registered image is then meshed to obtain a three-dimensional mesh; the initial surface fluorescence distribution image is mapped onto the surface of the three-dimensional mesh to obtain the target surface fluorescence distribution image; the overlapping region of the three-dimensional mesh and the target surface fluorescence distribution image is masked to obtain a target mask image; the target surface fluorescence distribution image is reconstructed using a target reconstruction function to obtain an initial fluorescence molecular tomographic image; the structural prior features corresponding to the target mask image, the initial features corresponding to the initial fluorescence molecular tomographic image, and the optical features corresponding to the target surface fluorescence distribution image are extracted; the structural prior features, density spatial features, and optical features are fused based on an attention mechanism to obtain the target fluorescence molecular tomographic image.

[0114] Based on the above image determination method, the present invention also provides an image determination apparatus. The following will be combined with... Figure 6 The device is described in detail.

[0115] Figure 6 A structural block diagram of an image determination apparatus according to an embodiment of the present invention is shown.

[0116] like Figure 6 As shown, the image determination device 600 of this embodiment includes a masking module 610, an optimization module 620, a feature extraction module 630, and a fusion module 640.

[0117] The masking module 610 is used to mask the overlapping area of ​​the anatomical image and the fluorescence distribution image of the target body surface to obtain a target mask image for marking the tumor region of the target object. The fluorescence distribution image of the target body surface represents the fluorescence intensity distribution of fluorescent molecules overflowing from the surface of the target object in three-dimensional space, and the anatomical image represents the tissue structure of the target object.

[0118] The optimization module 620 is used to reconstruct the fluorescence distribution image of the target body surface using the target reconstruction function to obtain an initial fluorescence molecule tomographic image, which characterizes the initial density spatial distribution of fluorescent molecules in the tissue.

[0119] The feature extraction module 630 is used to extract the prior structural features corresponding to the target mask image, the density spatial features corresponding to the initial fluorescence molecular tomographic image, and the optical features corresponding to the fluorescence distribution image on the target surface.

[0120] The fusion module 640 is used to fuse prior structural features, density spatial features, and optical features based on an attention mechanism to obtain a tomographic image of the target fluorescent molecule.

[0121] Optionally, the mask module 610 includes a first mask submodule, a second mask submodule, a third mask submodule, and a fourth mask submodule.

[0122] The first masking submodule is used to perform pixel registration between the anatomical image and the initial surface fluorescence distribution image to obtain a registered image. The initial surface fluorescence distribution image characterizes the fluorescence intensity distribution of fluorescent molecules overflowing from the surface of the target object in two-dimensional space.

[0123] The second mask submodule is used to perform meshing processing on the registered image to obtain a three-dimensional mesh, which represents the three-dimensional morphology of the target object's organizational structure.

[0124] The third masking submodule is used to map the initial surface fluorescence distribution image onto a three-dimensional mesh to obtain the target surface fluorescence distribution image.

[0125] The fourth masking submodule is used to mask the overlapping area of ​​the 3D mesh and the fluorescence distribution image on the target surface to obtain the target mask image.

[0126] Optionally, the fourth mask submodule includes a first mask unit and a second mask unit.

[0127] The first masking unit is used to process the fluorescence distribution image of the target body surface using an adaptive threshold algorithm to obtain the target region. The target region is the projection of the target fluorescence signal region in the fluorescence distribution image of the target body surface onto a three-dimensional space, and the intensity of the target fluorescence signal is greater than a preset threshold.

[0128] The second masking unit is used to mask the region in the 3D mesh that coincides with the target region to obtain the target mask image.

[0129] Optionally, the second mask submodule includes a third mask unit and a fourth mask unit.

[0130] The third mask unit is used to segment the registered image using a multimodal segmentation algorithm to obtain the initial mask image.

[0131] The fourth mask unit is used to perform meshing processing on the initial mask image to obtain a three-dimensional mesh.

[0132] Optionally, optimization module 620 includes an iterative submodule.

[0133] The iterative submodule is used to iteratively reconstruct the target surface fluorescence distribution image based on the conjugate gradient algorithm and target constraints to obtain an initial fluorescence molecular tomographic image. The conjugate gradient algorithm is used to optimize the fluorescence intensity distribution of the fluorescence molecular tomographic image reconstructed based on the target surface fluorescence distribution image in the tumor region. The target constraints are used to ensure that the fluorescence intensity distribution of the predicted surface fluorescence distribution image is close to that of the target surface fluorescence distribution image. The predicted surface fluorescence distribution image is obtained by transforming the reconstructed fluorescence molecular tomographic image based on a weight matrix. The weight matrix represents the mapping relationship between the surface fluorescence distribution image and the fluorescence molecular tomographic image constructed based on the optical transmission model.

[0134] Optionally, the fusion module 640 includes a first fusion submodule, a first correction submodule, a second correction submodule, and a second fusion submodule.

[0135] The first fusion submodule is used to perform multi-scale fusion of structural prior features, density space features and optical features to obtain multi-scale features.

[0136] The first correction submodule is used to perform structural correction on multi-scale features based on structural prior features to obtain structurally corrected features.

[0137] The second correction submodule is used to spatially correct the structural correction features based on the density space features, so as to obtain the spatial correction features.

[0138] The second fusion submodule is used to process spatial correction features using a multi-head attention mechanism to obtain tomographic images of the target fluorescent molecules.

[0139] Optionally, the second fusion submodule includes a fusion unit, a modulation unit, a deconvolution unit, a fully connected unit, and a mapping unit.

[0140] The fusion unit is used to perform attention fusion on the value features, key features and query features determined based on spatial correction features using a multi-head attention mechanism to obtain attention weights.

[0141] The modulation unit is used to modulate the spatially corrected features using attention weights to obtain modulated features.

[0142] The deconvolution unit is used to extract deconvolution features from the modulation features to obtain deconvolution features.

[0143] Fully connected units are used to perform fully connected processing on deconvolutional features to obtain fully connected features.

[0144] The mapping unit is used to map fully connected features back to three-dimensional space to obtain a tomographic image of the target fluorescent molecule.

[0145] Optionally, any plurality of modules among the masking module 610, optimization module 620, feature extraction module 630, and fusion module 640 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. Optionally, at least one of the masking module 610, optimization module 620, feature extraction module 630, and fusion module 640 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the masking module 610, optimization module 620, feature extraction module 630, and fusion module 640 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0146] Figure 7 A block diagram of an electronic device suitable for implementing an image determination method according to an embodiment of the present invention is shown.

[0147] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0148] like Figure 7 As shown, a computer electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a ROM 702 (read-only memory) or a program loaded from a storage portion 708 into a RAM 703 (random access memory). The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0149] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 702 and / or RAM 703. It should be noted that programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0150] Optionally, the electronic device 700 may also include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0151] Optionally, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined in the system of embodiments of the present invention. Optionally, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0152] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the image determination method according to embodiments of the present invention.

[0153] Optionally, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0154] For example, optionally, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0155] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the image determination method provided in the embodiments of the present invention.

[0156] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this embodiment of the invention. Optionally, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0157] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0158] Optionally, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or pairings fall within the scope of this invention.

[0160] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. An image determination method, characterized in that, The method includes: The overlapping areas of the anatomical image and the fluorescence distribution image of the target body surface are masked to obtain a target mask image for marking the tumor region of the target object. The fluorescence distribution image of the target body surface represents the fluorescence intensity distribution of fluorescent molecules overflowing from the surface of the target object in three-dimensional space, and the anatomical image represents the tissue structure of the target object. The fluorescence distribution image on the target body surface is reconstructed using the target reconstruction function to obtain an initial fluorescence molecule tomographic image, which characterizes the initial density spatial distribution of fluorescent molecules in the tissue. Extract the structural prior features corresponding to the target mask image, the density spatial features corresponding to the initial fluorescence molecular tomography image, and the optical features corresponding to the fluorescence distribution image on the target surface; Based on the attention mechanism, the structural prior features, the density spatial features, and the optical features are fused to obtain the target fluorescent molecule tomographic image.

2. The method according to claim 1, characterized in that, The overlapping areas of the anatomical image and the surface fluorescence distribution image are masked to obtain the target mask image, including: The anatomical image and the initial surface fluorescence distribution image are pixel-registered to obtain a registered image. The initial surface fluorescence distribution image characterizes the fluorescence intensity distribution of fluorescent molecules overflowing from the surface of the target object in two-dimensional space. The registered image is processed into a grid to obtain a three-dimensional grid, which represents the three-dimensional morphology of the target object's organizational structure. The initial surface fluorescence distribution image is mapped onto the three-dimensional grid to obtain the target surface fluorescence distribution image; The overlapping area of ​​the three-dimensional mesh and the fluorescence distribution image of the target surface is masked to obtain a target mask image.

3. The method according to claim 2, characterized in that, The step of masking the overlapping region of the three-dimensional mesh and the fluorescence distribution image of the target surface to obtain a target mask image includes: The target body surface fluorescence distribution image is processed using an adaptive threshold algorithm to obtain the target region, which is the projection of the target fluorescence signal region in the target body surface fluorescence distribution image onto a three-dimensional space, and the intensity of the target fluorescence signal is greater than a preset threshold. The regions in the three-dimensional mesh that overlap with the target region are masked to obtain a target mask image.

4. The method according to claim 2, characterized in that, The step of performing meshing processing on the registered image to obtain a three-dimensional mesh includes: The registered image is segmented using a multimodal segmentation algorithm to obtain an initial mask image; The initial mask image is meshed to obtain a three-dimensional mesh.

5. The method according to claim 1, characterized in that, The initial reconstruction of the fluorescence distribution image on the target surface using the target reconstruction function to obtain an initial fluorescence molecular tomographic image includes: Based on the conjugate gradient algorithm and target constraints, the fluorescence distribution image of the target surface is iteratively reconstructed to obtain an initial fluorescence molecular tomographic image. The conjugate gradient algorithm is used to optimize the fluorescence intensity distribution of the fluorescent molecular tomographic image reconstructed from the target surface fluorescence distribution image in the tumor region. The target constraint condition is used to constrain the fluorescence intensity distribution of the predicted surface fluorescence distribution image to be close to the fluorescence intensity distribution of the target surface fluorescence distribution image. The predicted surface fluorescence distribution image is obtained by transforming the reconstructed fluorescent molecular tomographic image based on a weight matrix. The weight matrix represents the mapping relationship between the surface fluorescence distribution image and the fluorescent molecular tomographic image constructed based on the optical transmission model.

6. The method according to claim 5, characterized in that, The target constraint conditions include at least a sparse regularization penalty term and a smoothing penalty term. The sparse regularization penalty term is used to constrain the fluorescence intensity distribution of the reconstructed fluorescent molecular tomographic image in the non-tumor region to meet the sparsity condition. The smoothing penalty term is used to constrain the smoothness of the fluorescence intensity distribution of the reconstructed fluorescent molecular tomographic image in the tumor region to be greater than a preset smoothness threshold.

7. The method according to claim 1, characterized in that, The process of fusing the structural prior features, the density spatial features, and the optical features based on an attention mechanism to obtain a tomographic image of the target fluorescent molecule includes: Multi-scale features are obtained by fusing structural prior features, density spatial features, and optical features at multiple scales. Based on the prior structural features, the multi-scale features are structurally corrected to obtain structurally corrected features; Based on the density space characteristics, the structural modification characteristics are spatially modified to obtain spatially modified characteristics; The spatial correction features are processed using a multi-head attention mechanism to obtain a tomographic image of the target fluorescent molecule.

8. The method according to claim 7, characterized in that, The process of using a multi-head attention mechanism to process the spatially corrected features to obtain a tomographic image of the target fluorescent molecule includes: A multi-head attention mechanism is used to fuse the value features, key features, and query features determined based on the spatial correction features to obtain attention weights. The spatially corrected features are modulated using the attention weights to obtain modulated features; The modulation features are subjected to deconvolution feature extraction to obtain deconvolution features; The deconvolutional features are processed by a fully connected layer to obtain fully connected features. The fully connected features are mapped back to three-dimensional space to obtain a tomographic image of the target fluorescent molecule.

9. An image determining device, characterized in that, The device includes: The masking module is used to mask the overlapping area of ​​the anatomical image and the fluorescence distribution image of the target body surface to obtain a target mask image for marking the tumor region of the target object. The fluorescence distribution image of the target body surface represents the fluorescence intensity distribution of fluorescent molecules overflowing from the surface of the target object in three-dimensional space, and the anatomical image represents the internal tissue structure of the target object. The optimization module is used to reconstruct the fluorescence distribution image of the target body surface using the target reconstruction function to obtain an initial fluorescence molecule tomographic image, wherein the initial fluorescence molecule tomographic image characterizes the initial density spatial distribution of fluorescent molecules in the tissue. The feature extraction module is used to extract the structural prior features corresponding to the target mask image, the density spatial features corresponding to the initial fluorescence molecular tomography image, and the optical features corresponding to the fluorescence distribution image on the target surface. The fusion module is used to fuse the structural prior features, the density spatial features, and the optical features based on an attention mechanism to obtain a tomographic image of the target fluorescent molecule.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images