Artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method and system

Abstract

The application belongs to the field of medical image processing, and relates to a medical image lesion segmentation and three-dimensional reconstruction method and system based on artificial intelligence, aiming to solve the problem of low model precision caused by mutual isolation of segmentation and reconstruction and one-way error transmission. The method comprises the following steps: fusing multi-modal medical images; generating a preliminary lesion mask using a segmentation network; constructing an initial three-dimensional geometric surface based on the mask and performing physically driven surface optimization; projecting the optimized model to the feature space of the segmentation network in reverse, calculating the spatial inconsistency thereof with the network prediction, and generating an attention weight map; feeding the attention weight map back to the segmentation network, iteratively updating the network parameters to refine the segmentation boundary; and based on the final segmentation result after convergence, performing three-dimensional reconstruction guided by network features again. The application constructs a closed-loop feedback and collaborative optimization mechanism between segmentation and reconstruction, significantly improving the accuracy of lesion segmentation and the geometric fidelity of the three-dimensional reconstruction model.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and specifically relates to a method and system for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence. Background Technology

[0002] Precise segmentation and three-dimensional visualization reconstruction of lesions in medical imaging (such as CT, MRI, PET, etc.) are key technologies in modern precision medicine. High-precision segmentation results form the basis for quantitative analysis, surgical planning, efficacy evaluation, and disease progression research, while high-quality three-dimensional reconstruction models provide doctors with intuitive three-dimensional anatomical relationships and lesion morphology.

[0003] Currently, deep learning-based automatic segmentation methods have become the mainstream research approach. A typical technical path is usually sequential: first, the acquired multimodal images are registered and preprocessed; second, one or a series of segmentation networks, such as U-Net and its variants, are used to directly predict lesion masks on 2D slices or 3D volume data; finally, based on the segmented binary masks, 3D surface reconstruction is performed using algorithms such as the Moving Cube, generating a visualized model. However, this linear processing paradigm of segmentation followed by reconstruction has several inherent drawbacks:

[0004] First, the segmentation and reconstruction processes are isolated from each other, lacking a collaborative optimization mechanism. Segmentation networks are typically trained solely on pixel-level or voxel-level classification accuracy, without explicitly considering the 3D geometric plausibility that their outputs should satisfy, such as surface smoothness and topological correctness. Therefore, errors generated by the segmentation network, such as boundary burrs, isolated noise points, and holes, are directly passed to the reconstruction stage without correction, resulting in a 3D model containing numerous geometric flaws that fail to faithfully reflect the true biomechanical morphology of the lesions.

[0005] Secondly, the 3D reconstruction process lacks high-level semantic guidance. Traditional contour extraction algorithms rely solely on the binary information of the segmentation mask. Their smoothing or optimization processes are blind, low-level optimizations that completely disregard the rich texture and contextual features of the image data itself. This makes it impossible to ensure that the reconstructed surface is semantically consistent with the actual boundary of the lesion in the original image.

[0006] Furthermore, segmentation networks cannot benefit from high-quality 3D geometric information. Existing methods lack an effective mechanism to encode the reconstructed 3D geometric information and feed it back into the segmentation network to correct its segmentation bias.

[0007] Therefore, existing technologies essentially treat segmentation and reconstruction as two independent, sequentially executed modules, leading to unidirectional accumulation and propagation of errors, and the two modules cannot complement each other's advantages. There is an urgent need for a new integrated method that can achieve bidirectional driving, closed-loop iteration, and collaborative evolution of lesion segmentation and 3D reconstruction. Summary of the Invention

[0008] To address the aforementioned problems in the prior art, namely the isolation between segmentation and reconstruction processes and the low model accuracy caused by unidirectional error propagation, this invention provides a method and system for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence.

[0009] In a first aspect, this invention proposes an artificial intelligence-based method for medical image lesion segmentation and three-dimensional reconstruction, the method comprising:

[0010] Acquire multimodal medical images and perform registration and fusion to generate unified three-dimensional volume data;

[0011] The unified 3D volume data is input into the first segmentation sub-network of the cascaded segmentation network to obtain the primary segmentation result; the primary segmentation result is combined with the unified 3D volume data by feature concatenation and input into the second segmentation sub-network of the cascaded segmentation network to generate a binarized 3D lesion mask;

[0012] Based on the binarized 3D lesion mask, lesion voxels are extracted, and an initial 3D geometric surface is constructed through adaptive mesh discretization and vertex optimization.

[0013] The initial three-dimensional geometric surface is converted into an implicit function representation, and iterative optimization is performed through physics-driven surface evolution constraints to obtain an optimized three-dimensional surface model.

[0014] The optimized 3D surface model is back-projected onto the feature space of the second segmentation sub-network, and an attention weight map is generated based on spatial inconsistency.

[0015] The attention weight map is fed back to the second segmentation sub-network to iteratively update the network parameters and converge the segmentation boundary, and the final segmentation result is output.

[0016] Based on the final segmentation result, three-dimensional reconstruction is performed again, and surface optimization is performed using geometric consistency constraints derived from network features to generate the final three-dimensional reconstruction model.

[0017] Furthermore, multimodal medical images are acquired, registered, and fused to generate unified three-dimensional volumetric data. The method is as follows:

[0018] At least two three-dimensional medical image sequences of the same subject are acquired, and each three-dimensional medical image sequence is preprocessed to unify spatial resolution and normalize grayscale intensity.

[0019] The non-reference image sequence is downsampled to obtain a low-resolution image. The initial three-dimensional deformation field is estimated based on the low-resolution image. Then, the initial three-dimensional deformation field is iteratively optimized at the original resolution to obtain the final three-dimensional deformation field.

[0020] Based on the final three-dimensional deformation field, nonlinear spatial registration is performed on each sequence, and multimodal voxel features are fused using a channel attention mechanism within the registration space to generate the unified three-dimensional volume data.

[0021] Furthermore, the method for feature concatenating the primary segmentation result with the unified 3D volume data is as follows:

[0022] The initial segmentation result is processed into single-channel probabilistic volume data with the same spatial dimension as the unified three-dimensional volume data;

[0023] The single-channel probability volume data is concatenated with the unified three-dimensional volume data along the channel dimension to form enhanced feature volume data;

[0024] The enhanced feature volume data is fused by convolutional processing along the channel dimension to generate the input of the second segmentation sub-network.

[0025] Furthermore, based on the binarized 3D lesion mask, lesion voxels are extracted, and an initial 3D geometric surface is constructed through adaptive mesh discretization and vertex optimization. The method is as follows:

[0026] Extract the set of voxels marked as lesions by the binarized three-dimensional lesion mask from the unified three-dimensional volume data;

[0027] Using the outer boundary voxel of the voxel set as the center, generate tetrahedral mesh elements of adaptive size to discretize the three-dimensional space inside the lesion;

[0028] For each voxel in the voxel set, calculate the gray-level gradient direction and magnitude of all voxels in the three-dimensional spatial neighborhood of the voxel.

[0029] For any vertex in the tetrahedral mesh cell located in the boundary region of the binary three-dimensional lesion mask, select the voxel position with the largest gray-scale gradient magnitude from the three-dimensional spatial neighborhood where the vertex is located, and update the spatial coordinates of the vertex to the selected voxel position.

[0030] After traversing and optimizing the vertices of all boundary regions, the initial three-dimensional geometric surface is formed.

[0031] Furthermore, in generating tetrahedral mesh elements with adaptive dimensions, the method for calculating the adaptive dimensions is as follows:

[0032] Obtain the grayscale gradient magnitude of the target voxel and all voxels in its spatial neighborhood;

[0033] Calculate the average value and standard deviation of the gray-level gradient magnitude within the spatial neighborhood;

[0034] Based on a preset reference size constant, the side length of the tetrahedral mesh cell to be generated at the target voxel is calculated by subtracting the ratio of the sum of the average value and the standard deviation from the reference size constant.

[0035] The ratio is the product of the sum of the average and the standard deviation and a preset fixed constant.

[0036] Furthermore, the initial three-dimensional geometric surface is converted into an implicit function representation, and iterative optimization is performed through physics-driven surface evolution constraints to obtain an optimized three-dimensional surface model. The method is as follows:

[0037] An axial bounding box is constructed that completely contains the initial three-dimensional geometric surface, and the internal space of the bounding box is regularly divided into a three-dimensional voxel mesh. The shortest directed distance from the center of each voxel to the initial three-dimensional geometric surface is calculated, and an initial directed distance field is generated as an implicit function.

[0038] Construct an energy functional consisting of an integral term of the square of the mean curvature of the surface and a penalty term for changes in the surface area;

[0039] The energy functional is minimized using the numerical gradient descent method. In each iteration, the distance values ​​of each voxel are updated based on the spatial gradient of the current directed distance field and the variational derivative of the energy functional.

[0040] The iteration stops when the change in the value of the energy functional between two consecutive iterations is less than one ten-thousandth of its value before the first iteration.

[0041] Zero-value isosurfaces are extracted from the final directed distance field and converted into triangular meshes to generate the optimized 3D surface model.

[0042] Furthermore, the optimized 3D surface model is back-projected into the feature space of the second segmentation sub-network, and an attention weight map is generated based on spatial inconsistency. The method is as follows:

[0043] Obtain the target 3D feature map output by the second segmentation sub-network when processing the enhanced feature volume data;

[0044] The vertex coordinates of the optimized 3D surface model are transformed into the spatial coordinate system corresponding to the target 3D feature map;

[0045] On the target 3D feature map, find the voxel position closest to the coordinates of each transformed vertex, and extract the multi-channel feature vector at that voxel position.

[0046] Based on the multi-channel feature vector, calculate the feature response value corresponding to each vertex;

[0047] The absolute value of the difference between the value of the binarized three-dimensional lesion mask at the same vertex position and the feature response value is calculated as a measure of spatial inconsistency at that vertex.

[0048] Based on the spatial inconsistency measure at all vertices, an attention weight map with the same spatial size as the target 3D feature map is generated by 3D interpolation.

[0049] Furthermore, based on the final segmentation result, 3D reconstruction is performed again, and surface optimization is performed using geometric consistency constraints derived from network features to generate the final 3D reconstruction model. The method is as follows:

[0050] Based on the final segmentation result, the corresponding set of lesion voxels is extracted from the unified three-dimensional volume data, and mesh discretization and vertex optimization are performed to construct a secondary initial three-dimensional geometric surface.

[0051] The quadratic initial three-dimensional geometric surface is converted into an implicit function representation;

[0052] Obtain the target 3D feature map generated by the second segmentation sub-network during the output of the final segmentation result;

[0053] Based on the target 3D feature map, calculate the distribution divergence of multi-channel feature vectors at each voxel position, and construct a geometric consistency term to constrain the surface evolution;

[0054] The geometric consistency term is combined with the curvature smoothing term and the area constraint term in a weighted manner to form a new energy functional.

[0055] The new energy functional is minimized using the numerical gradient descent method to optimize the implicit function;

[0056] The zero-value isosurface is extracted from the optimized implicit function and converted into a triangular mesh to generate the final 3D reconstruction model.

[0057] Furthermore, based on the target 3D feature map, the distribution divergence of the multi-channel feature vectors at each voxel location is calculated, and a geometric consistency term for constraining the surface evolution is constructed. The method is as follows:

[0058] For each voxel position in the target 3D feature map, extract the multi-channel feature vector of all voxels in the 3D neighborhood centered on that voxel position;

[0059] Calculate the covariance matrix of all multi-channel feature vectors within the three-dimensional neighborhood;

[0060] Solve for the eigenvalues ​​of the covariance matrix, and use the sum of the eigenvalues ​​as the feature distribution divergence value of the central voxel position;

[0061] Based on the feature distribution divergence values ​​at all voxel locations, the feature distribution divergence values ​​are mapped to the corresponding geometric constraint weights through a preset monotonically decreasing function;

[0062] Using the geometric constraint weights, the geometric consistency term is constructed, which is the integral of the geometric constraint weights and the square of the implicit function gradient magnitude over the surface domain.

[0063] In a second aspect, the present invention proposes an artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction system, and an artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method, the system comprising:

[0064] The image acquisition and fusion module is configured to acquire multimodal medical images, perform registration and fusion, and generate unified three-dimensional volume data;

[0065] The cascaded segmentation module is configured to input the unified 3D volume data into the first segmentation sub-network of the cascaded segmentation network to obtain a primary segmentation result; to perform feature concatenation between the primary segmentation result and the unified 3D volume data, and input the result into the second segmentation sub-network of the cascaded segmentation network to generate a binarized 3D lesion mask.

[0066] The three-dimensional surface initial construction module is configured to extract lesion voxels based on the binarized three-dimensional lesion mask and construct the initial three-dimensional geometric surface through adaptive mesh discretization and vertex optimization.

[0067] The surface optimization module is configured to convert the initial three-dimensional geometric surface into an implicit function representation and perform iterative optimization through physical-driven surface evolution constraints to obtain an optimized three-dimensional surface model.

[0068] The attention map generation module is configured to back-project the optimized 3D surface model to the feature space of the second segmentation sub-network and generate an attention weight map based on spatial inconsistency.

[0069] The feedback training module is configured to feed the attention weight map back to the second segmentation sub-network, iteratively update the network parameters and converge the segmentation boundary, and output the final segmentation result.

[0070] The final 3D reconstruction module is configured to perform 3D reconstruction again based on the final segmentation result, and perform surface optimization with geometric consistency constraints derived from network features to generate the final 3D reconstruction model.

[0071] The beneficial effects of this invention are:

[0072] This invention establishes a closed-loop iterative collaborative optimization framework by back-projecting an optimized 3D geometric model to generate an attention map and feeding it back to the segmentation network. This allows the geometric constraints in the 3D reconstruction process to guide and refine the segmentation network's predictions. Simultaneously, the deep semantic information provided by the segmentation network guides the generation of the final 3D model, achieving complementary advantages and co-evolution of the two tasks, fundamentally solving the problem of unidirectional error accumulation in existing technologies.

[0073] This invention introduces a 3D model optimized through physical-driven surface evolution as a shape prior. This effectively corrects geometric imperfections such as boundary spikes, internal holes, and isolated noise points generated by the segmentation network in the initial prediction. The attention feedback mechanism guides the network to focus its attention on geometrically inconsistent regions, thereby outputting a final segmentation result with more accurate topology and smoother boundaries, improving the overall quality of the segmentation mask.

[0074] In the final reconstruction stage, this invention utilizes the distribution divergence of deep features in the segmentation network to construct geometric consistency constraints. These constraints enable the surface optimization process to perceive the semantic information of the image, preserving details in areas with drastic feature changes, such as lesion boundaries, while smoothing flat internal regions. Compared to the blind geometric smoothing of traditional methods, the final 3D model generated by this method more faithfully reflects the true morphology and boundary features of lesions in the original image.

[0075] This invention employs adaptive mesh discretization and an image gradient-based vertex optimization strategy to construct the initial 3D surface. Adaptive mesh density adjustment allows for the use of finer meshes at complex lesion boundaries, efficiently capturing details. The vertex optimization step precisely snaps surface vertices to the locations of the strongest gradients in the image, ensuring the initial model already possesses high boundary fit, laying a solid foundation for subsequent optimization and feedback. Attached Figure Description

[0076] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0077] Figure 1 This is a schematic diagram of the overall process of a medical image lesion segmentation and three-dimensional reconstruction method based on artificial intelligence according to the present invention;

[0078] Figure 2This is a schematic diagram illustrating the optimized generation of a three-dimensional surface model in an artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method of the present invention.

[0079] Figure 3 This is a schematic diagram of attention weight map generation in an artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method of the present invention;

[0080] Figure 4 This is a schematic diagram illustrating the generation of the final three-dimensional reconstruction model in an artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method of the present invention.

[0081] Figure 5 This is a schematic diagram of the structural connection of an artificial intelligence-based medical imaging lesion segmentation and three-dimensional reconstruction system according to the present invention. Detailed Implementation

[0082] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0083] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0084] The first embodiment of the present invention provides a method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence, the method comprising:

[0085] Step S10: Acquire multimodal medical images and perform registration and fusion to generate unified three-dimensional volume data;

[0086] Step S20: Input the unified three-dimensional volume data into the first segmentation sub-network of the cascaded segmentation network to obtain the primary segmentation result; perform feature concatenation between the primary segmentation result and the unified three-dimensional volume data, and input the result into the second segmentation sub-network of the cascaded segmentation network to generate a binarized three-dimensional lesion mask.

[0087] Step S30: Extract lesion voxels based on the binarized three-dimensional lesion mask, and construct the initial three-dimensional geometric surface through adaptive mesh discretization and vertex optimization;

[0088] Step S40: The initial three-dimensional geometric surface is converted into an implicit function representation and iteratively optimized through physical-driven surface evolution constraints to obtain an optimized three-dimensional surface model;

[0089] Step S50: The optimized three-dimensional surface model is back-projected into the feature space of the second segmentation sub-network, and an attention weight map is generated based on the spatial inconsistency.

[0090] Step S60: Feed the attention weight map back to the second segmentation sub-network, iteratively update the network parameters and make the segmentation boundary converge, and output the final segmentation result;

[0091] Step S70: Perform 3D reconstruction again based on the final segmentation result, and optimize the surface using geometric consistency constraints derived from network features to generate the final 3D reconstruction model.

[0092] To more clearly illustrate the artificial intelligence-based medical image lesion segmentation and 3D reconstruction method of this invention, the following will be combined with... Figure 1 The steps in the embodiments of the present invention are described in detail below:

[0093] Step S10: Acquire multimodal medical images and perform registration and fusion to generate unified three-dimensional volume data;

[0094] In this embodiment, step S10 includes:

[0095] Step S11: Obtain at least two three-dimensional medical image sequences of the same subject, and perform spatial resolution unification and grayscale intensity normalization preprocessing on each three-dimensional medical image sequence.

[0096] Step S12: Downsample the non-reference image sequence to obtain a low-resolution image. Estimate the initial three-dimensional deformation field based on the low-resolution image. Then, iteratively optimize the initial three-dimensional deformation field at the original resolution to obtain the final three-dimensional deformation field.

[0097] Step S13: Perform nonlinear spatial registration on each sequence based on the final three-dimensional deformation field, and use a channel attention mechanism to fuse multimodal voxel features within the registration space to generate the unified three-dimensional volume data.

[0098] In this embodiment, step S11 is first executed to acquire at least two three-dimensional medical image sequences of the same subject acquired at similar time points, such as a computed tomography (CT) sequence and an magnetic resonance imaging (MRI) sequence. Specifically, the MRI sequence can be a T1-weighted contrast-enhanced T1CE sequence, which is advantageous for displaying the boundaries of soft tissue lesions. After acquiring the original images, each three-dimensional medical image sequence needs to undergo standardized preprocessing to eliminate differences caused by different devices and scanning parameters. This preprocessing includes two steps: spatial resolution unification and grayscale intensity normalization. In spatial resolution unification, one modality, such as CT images, is selected as the reference space, and all other modalities of images, i.e., the MRI images in this case, are resampled using a trilinear interpolation algorithm to ensure that their voxel spacing is completely consistent with the voxel spacing of the reference CT image, for example, uniformly set to an isotropic resolution of 1mm × 1mm × 1mm, thereby ensuring that all images have the same sampling grid in physical space. Subsequently, grayscale intensity normalization is performed independently on the image data of all modalities. To enhance the robustness of the deep learning model, this embodiment employs the Z-score normalization method. Specifically, it calculates the mean and standard deviation of grayscale values ​​for all voxels within each 3D image, then subtracts the mean from the grayscale value of each voxel and divides by the standard deviation. To suppress the influence of extreme outliers, grayscale values ​​can be cropped to, for example, the range defined by the 0.5 percentile and 99.5 percentile before calculating the mean and standard deviation.

[0099] After preprocessing, step S12 is executed. The goal of this step is to obtain a high-precision final three-dimensional deformation field to accurately align anatomical structures in different modalities. Since performing nonlinear registration directly on the original high-resolution images is computationally intensive and prone to getting trapped in local optima, this embodiment adopts a coarse-to-fine multi-resolution registration strategy. Specifically, the non-reference image sequence to be registered, i.e., the MRI images, is smoothed using Gaussian smoothing and then downsampled with a step size of 2 to generate one or more low-resolution image pyramid levels. First, on the lowest resolution image, an initial and relatively smooth three-dimensional deformation field is estimated by optimizing a cost function composed of an image similarity metric and a regularization term. Here, the image similarity metric can be normalized mutual information, which is insensitive to the linear relationship of gray levels between different modalities, while the regularization term can be the bending energy of a B-spline transform model. This initial deformation field describes the approximate transformation relationship mapping the MRI image space to the CT image space. Subsequently, this initial deformation field is upsampled to a higher image resolution through interpolation. Starting from this point, the cost function is iteratively optimized at this resolution to finely adjust the deformation field. This process is repeated step by step until the final round of optimization iterations is completed at the original full resolution, ultimately obtaining a final three-dimensional deformation field that can accurately capture differences in local anatomical structures.

[0100] Finally, step S13 is executed, whereby nonlinear spatial registration is performed on each sequence based on the obtained final three-dimensional deformation field, and multimodal features are fused to generate unified three-dimensional volume data. Specifically, the final three-dimensional deformation field is applied to the non-reference MRI image, and a deformed MRI image that is spatially aligned with the reference CT image at the voxel level is generated through trilinear interpolation. At this point, the CT image and the deformed MRI image in the same coordinate system are obtained. To effectively integrate the complementary information of these two modalities into a single input, this embodiment employs a feature fusion method based on a channel attention mechanism. The spatially aligned CT volume data and the deformed MRI volume data are stitched together along the channel dimension to form a multi-channel three-dimensional tensor; in this example, it is a 2-channel tensor.

[0101] This tensor is input into a 3D channel attention module, which first performs global average pooling on the spatial dimensions (height, width, and depth), compressing the global spatial information of each channel into a single value, thus obtaining a channel descriptor vector. This vector is then nonlinearly transformed through a bottleneck structure containing two fully connected layers. The first fully connected layer uses the ReLU activation function for dimensionality reduction, and the second fully connected layer restores the dimensions. Finally, a sigmoid activation function outputs a weight vector of the same length as the number of channels. Each value in this weight vector is between 0 and 1, representing the importance of the CT and MRI channels, respectively.

[0102] Finally, the learned weight vector is multiplied channel-by-channel with the original multi-channel 3D tensor. This operation is typically achieved using a broadcast mechanism, thereby completing the adaptive weighting of features from different modalities. After this fusion, a unified 3D volumetric data with the same spatial dimensions as the original image but with intelligently recalibrated channel features is generated. This data retains all the information from the multimodalities and enhances the representation of features more valuable to the current segmentation task through an attention mechanism, making it directly usable as input to subsequent cascaded segmentation networks.

[0103] To further illustrate this process, a specific glioma segmentation task is used as an example. In step S11, the patient's CT images are acquired, with an original resolution of 512×512×120 voxels and a voxel spacing of 0.7mm×0.7mm×1.5mm; simultaneously, T1CE sequence MRI images are acquired, with an original resolution of 256×256×160 voxels and a voxel spacing of 1.0mm×1.0mm×1.0mm. Using the MRI space as a reference, the CT images are resampled to an isotropic resolution of 1.0mm using trilinear interpolation and then grayscale normalized. In step S12, the resampled CT images are downsampled two levels to obtain low-resolution images of half and 1 / 4 of the original size, respectively. First, the initial global deformation field is estimated at the 1 / 4 resolution, then it is upsampled and optimized at the 1 / 2 resolution, and finally fine-tuned at the original 1.0mm resolution to obtain the final deformation field. In step S13, this final deformation field is applied to the normalized MRI image to obtain MRI volume data perfectly aligned with the CT space. These two volume data are then concatenated along the channel dimension to form a 2-channel 3D tensor with dimensions of 256×256×160. This tensor, after passing through a channel attention module, generates a weight vector such as 0.4 and 0.9, indicating that the network considers the information from the T1CE channel more important for identifying tumor boundaries than the information from the CT channel, and therefore assigns it a higher weight. Finally, all voxel values ​​from the original CT channel are multiplied by 0.4, and all voxel values ​​from the T1CE channel are multiplied by 0.9, resulting in a new 2-channel 3D tensor, which serves as the unified 3D volume data input to the segmentation network.

[0104] Step S20: Input the unified three-dimensional volume data into the first segmentation sub-network of the cascaded segmentation network to obtain the primary segmentation result; perform feature concatenation between the primary segmentation result and the unified three-dimensional volume data, and input the result into the second segmentation sub-network of the cascaded segmentation network to generate a binarized three-dimensional lesion mask.

[0105] In this embodiment, the method for feature concatenating the primary segmentation result with the unified 3D volume data is as follows:

[0106] Step S21: The primary segmentation result is processed into single-channel probabilistic volume data with the same spatial dimension as the unified three-dimensional volume data.

[0107] Step S22: The single-channel probability volume data and the unified three-dimensional volume data are spliced ​​along the channel dimension to form enhanced feature volume data;

[0108] Step S23: Perform convolutional fusion on the channel dimension of the enhanced feature volume data to generate the input of the second segmentation sub-network.

[0109] In this embodiment, after generating unified 3D volume data in step S10, the invention proceeds to step S20, which employs a cascaded segmentation network structure to perform coarse-to-fine segmentation of the lesion. Specifically, the unified 3D volume data generated from multimodal image fusion in step S10 is first input into the first segmentation subnetwork of the cascaded segmentation network. In this embodiment, the first segmentation subnetwork can adopt the industry-known 3D U-Net structure, designed to quickly and approximate the lesion region. The network receives the unified 3D volume data as input, extracts high-level semantic features layer by layer through its encoder path, and simultaneously reduces the spatial resolution. Then, through skip connections between its decoder path and encoder path, it combines deep semantic information with shallow high-resolution detail features, gradually restoring the spatial resolution, and finally outputs a primary segmentation result with the same spatial dimension as the input volume data. This primary segmentation result is a probability map, where the value of each voxel is calculated using the Sigmoid activation function at the network's end, ranging from 0 to 1, representing the probability that the voxel belongs to the lesion region. This primary segmentation result provides strong spatial prior information for the second-stage fine segmentation.

[0110] To effectively provide this primary segmentation result to the second segmentation sub-network, it needs to be feature-concatenated with the original unified 3D volume data. This process is specifically implemented in this embodiment through steps S21, S22, and S23. First, step S21 is executed, processing the primary segmentation result into single-channel probability volume data with the same spatial dimension as the unified 3D volume data. Since the output of the first segmentation sub-network is itself a probability map of the same size as the input, this step mainly ensures the uniformity of the data format; that is, the primary segmentation result is treated as a single-channel 3D tensor, where each element represents the probability of lesion presence.

[0111] Then, step S22 is executed, where the single-channel probabilistic volume data is concatenated with the unified 3D volume data along the channel dimension to form enhanced feature volume data. For example, if the unified 3D volume data generated in step S10 is a 2-channel tensor containing CT and MRI information, with dimensions of 2×H×W×D, where H, W, and D are the spatial height, width, and depth, concatenating it with the single-channel probabilistic volume data of dimension 1×H×W×D will form an enhanced feature volume data of dimension 3×H×W×D. This newly formed data volume not only contains rich texture and intensity information of the original image, but also additionally obtains a probabilistic channel indicating the approximate location of the lesion, which can guide the subsequent network to concentrate computational resources more on areas where lesions may exist.

[0112] Finally, step S23 is executed to perform channel-dimensional convolutional fusion on the enhanced feature volume data, generating the final input to the second segmentation sub-network. Directly inputting the stitched multi-channel features into the network is feasible, but to achieve deeper feature interaction and fusion, this embodiment processes the enhanced feature volume data through one or more 1×1×1 convolutional layers before inputting it into the second segmentation sub-network. This 1×1×1 convolutional kernel can linearly combine and non-linearly activate features in the channel dimension without changing the spatial dimension, thereby learning the optimal fusion method between different channels, i.e., between the original image features and the primary segmentation probability. This convolutional fusion operation can intelligently extract the stitched 3-channel data into a feature volume data that is also 3-channel or has fewer or more channels, where each channel's feature is a weighted combination of the original information and spatial priors. This method is more flexible and efficient than simple feature stitching.

[0113] After the aforementioned feature concatenation and fusion, the final input data is fed into the second segmentation sub-network of the cascaded segmentation network. This second segmentation sub-network can also adopt a standard 3D U-Net structure, with an architecture identical to the first segmentation sub-network, but its network weights are trained independently because it undertakes the specific task of finely delineating the boundaries within the located regions. This network utilizes enhanced features fused with spatial priors to more accurately distinguish lesion tissue from surrounding healthy tissue, correcting segmentation errors that may occur in the first stage, such as removing false positive regions or filling segmentation gaps. The output of the second segmentation sub-network is also a refined 3D probability map. To obtain the final deterministic segmentation result, this probability map needs to be binarized to generate a binarized 3D lesion mask. This processing typically involves setting a threshold, such as 0.5, assigning a value of 1 to all voxels in the probability map with a value greater than 0.5, representing the lesion region; and assigning a value of 0 to the remaining voxels, representing the background region. The resulting binarized 3D lesion mask is the final output of the entire segmentation process and can be used for subsequent volume calculation, visualization, or treatment planning.

[0114] Step S30: Extract lesion voxels based on the binarized 3D lesion mask, and construct the initial 3D geometric surface through adaptive mesh discretization and vertex optimization.

[0115] Step S31: Extract the set of voxels marked as lesions by the binarized three-dimensional lesion mask from the unified three-dimensional volume data;

[0116] Step S32: Using the outer boundary voxel of the voxel set as the center, generate tetrahedral mesh units of adaptive size to discretize the three-dimensional space inside the lesion.

[0117] Step S33: For each voxel in the voxel set, calculate the gray-level gradient direction and magnitude of all voxels in the three-dimensional spatial neighborhood of the voxel.

[0118] Step S34: For any vertex in the tetrahedral mesh cell located in the boundary region of the binary three-dimensional lesion mask, select the voxel position with the largest gray-scale gradient magnitude from the three-dimensional spatial neighborhood where the vertex is located, and update the spatial coordinates of the vertex to the selected voxel position.

[0119] Step S35: After traversing and optimizing the vertices of all boundary regions, the initial three-dimensional geometric surface is formed.

[0120] In this embodiment, after accurately segmenting the lesion and generating a binary 3D lesion mask, the invention proceeds to step S30, the core objective of which is to transform the discrete, voxel-based segmentation results into a continuous, smooth, and anatomically more accurate 3D geometric surface model. This transformation is crucial for subsequent quantitative analysis, surgical planning, and high-fidelity visualization. This step is achieved through an adaptive mesh generation and image gradient-based vertex optimization process, specifically executed sequentially through steps S31 to S35.

[0121] The process begins in step S31, where all voxels marked as lesions are extracted from the unified 3D volume data generated in the previous steps, guided by the binarized 3D lesion mask. Specifically, the program iterates through each element of the 3D mask, recording the corresponding 3D spatial coordinates of any element with a value of 1, and retrieving the original multimodal feature values ​​for that coordinate location from the unified 3D volume data, such as CT values ​​and MRI signal intensity. This iterative process constructs a complete set of lesion voxels, which not only contains the spatial location information of the lesions but also associates them with their original imaging features, providing a data foundation for subsequent gradient calculations.

[0122] Next, step S32 is executed to discretize the three-dimensional space occupied by the lesion, i.e., to construct an initial mesh structure. This embodiment employs an adaptive tetrahedral meshing method. First, the outer boundary voxels in the voxel set are identified, i.e., those voxels that are themselves lesion voxels but whose three-dimensional neighborhood contains at least one background voxel. Initial mesh cells are generated with these outer boundary voxels as the center. To achieve adaptability, the size of the mesh cells can be adjusted according to the geometric complexity of the local surface. For example, in areas where the curvature of the mask surface changes drastically or where there are fine structures, smaller tetrahedral cells are generated to capture more details; while in areas inside the lesion or in areas with relatively flat surfaces, larger tetrahedral cells are generated to reduce computational redundancy. Through this method, a preliminary tetrahedral mesh that completely covers the lesion voxel set can be generated, with its outer surface roughly outlining the contour of the lesion. However, at this point, the position of the mesh vertices is still limited by the discrete characteristics of the voxel mesh, and there are obvious stepped defects on the surface.

[0123] To ensure the mesh surface accurately conforms to the actual anatomical boundaries, the mesh vertices need to be optimized using the grayscale information of the original image. The preparation for optimization is completed in step S33. This step calculates the grayscale gradient in the three-dimensional spatial neighborhood of each voxel in the lesion voxel set, using its grayscale information in the unified three-dimensional volume data. Specifically, methods such as the three-dimensional Sober operator or the Gaussian gradient operator can be used to calculate the voxel grayscale values ​​within a neighborhood window of, for example, 3×3×3 around each voxel, thereby obtaining a gradient direction vector representing the direction of the fastest grayscale change, and a gradient magnitude representing the degree of this change. Areas with larger gradient magnitudes typically correspond to the boundaries of different tissue types. By performing this operation on all lesion voxels, a three-dimensional gradient field covering the entire lesion region is pre-calculated and stored.

[0124] The core vertex optimization process is executed in step S34. The goal of this step is to move vertices located on the lesion surface in the initial mesh from their current discrete voxel positions to sub-voxel positions that better represent the true tissue boundary. Specifically, for any vertex in the tetrahedral mesh cell located in the boundary region of the binary 3D lesion mask, first, the 3D spatial neighborhood of that vertex is determined, for example, a cube region of a specific size centered at the vertex's current coordinates. Then, within this neighborhood, the grayscale gradient magnitudes of all voxels calculated in step S33 are retrieved, and the voxel with the largest gradient magnitude is selected. The position of this voxel is considered the point closest to the true tissue boundary within this neighborhood. Finally, the spatial coordinates of the currently processed mesh vertex are updated to the center coordinates of the selected voxel with the largest gradient.

[0125] In step S35, the entire surface is refined by traversing and optimizing all vertices located in the boundary region. Step S34 is repeated until all surface vertices of the initial tetrahedral mesh have completed position updates based on the maximum gradient magnitude. After this global optimization, the originally rough, stepped surface aligned with the voxel mesh is transformed into a smooth and precise 3D geometric surface. The vertices of this surface are no longer randomly located at the voxel centers, but are precisely snapped to the positions in the original image where grayscale changes are most dramatic, thus greatly improving the anatomical accuracy and visual realism of the final 3D model. The resulting initial 3D geometric surface is a standard 3D model defined by vertices and faces (usually triangles, composed of tetrahedral surfaces), which can be directly used for subsequent 3D rendering or further processing.

[0126] In this embodiment, in step S32, the method for calculating the adaptive size of the tetrahedral mesh element is as follows:

[0127] Step S321: Obtain the grayscale gradient magnitude of the target voxel and all voxels in the spatial neighborhood of the target voxel.

[0128] Step S322: Calculate the average value and standard deviation of the gray-level gradient magnitude within the spatial neighborhood;

[0129] Step S323: Based on a preset reference size constant, calculate the side length of the tetrahedral mesh unit to be generated at the target voxel by subtracting the ratio of the sum of the average value and the standard deviation from the reference size constant.

[0130] The ratio is the product of the sum of the average and the standard deviation and a preset fixed constant.

[0131] After extracting the complete set of lesion voxels in step S31, the present invention discretizes the three-dimensional space inside the lesion in step S32 by constructing an adaptive-sized tetrahedral mesh. The adaptive size here is key to achieving high-efficiency and high-fidelity modeling, and its specific calculation method is detailed in steps S321, S322, and S323 in this embodiment. The general idea of ​​this method is to generate fine, small-sized mesh units in areas with complex lesion boundary geometry, while generating coarse, large-sized mesh units in flat or internal lesion regions. In practice, the calculation of this adaptive size is performed independently for each target voxel that serves as the reference point for mesh generation, typically the outer boundary voxel of the lesion.

[0132] First, step S321 is executed. For any selected target voxel, the system needs to acquire the grayscale gradient magnitude of the target voxel and all voxels within its predetermined three-dimensional spatial neighborhood. The grayscale gradient magnitude data here originates from the three-dimensional gradient field pre-calculated and stored in the preceding step S34. In this embodiment, the spatial neighborhood is preferably a 3×3×3 cube region centered on the target voxel, containing 27 gradient magnitudes, which can fully reflect the image features within a very small area surrounding the target voxel. Through this step, the system collects a set of gradient information reflecting the image features of the region, particularly the boundary strength and texture complexity, at the core location of each mesh cell to be generated.

[0133] Next, step S322 is executed. Based on the set of gray-level gradient magnitudes of all voxels in the neighborhood obtained in the previous step, the mean and standard deviation of this set of values ​​are calculated. The mean reflects the average intensity of the tissue boundary within the local region; a higher mean usually indicates the presence of a clear and significant tissue boundary. The standard deviation, on the other hand, quantifies the dispersion or drastic change of the gradient magnitude. A higher standard deviation may indicate the presence of multiple subtle boundaries with varying orientations, or an extremely complex surface structure. Therefore, the combination of the mean and standard deviation constitutes an effective quantitative indicator of local geometric complexity.

[0134] In step S323, the final mesh cell size is calculated based on the aforementioned statistics. Specifically, based on a preset baseline size constant, a proportional value related to local complexity is subtracted from this baseline size constant to obtain the side length of the tetrahedral mesh cell to be generated at the target voxel. The baseline size constant represents the maximum allowable side length of the mesh cell; in this embodiment, this value is preferably set to 5.0 voxel units. The proportional value is obtained by multiplying the sum of the average and standard deviation calculated in step S322 by a preset fixed constant. This preset fixed constant serves as a sensitivity adjustment factor to control the influence weight of gradient information on the final size; in this embodiment, the fixed constant is preferably 0.01. For example, if the sum of the average and standard deviation of the gradient in a certain region is large, it indicates that the boundary of that region is clear and complex. The proportional value obtained after multiplying by the fixed constant is also correspondingly large. Subtracting this proportional value from the baseline size results in a smaller final side length, thus generating a finer mesh at that location. Conversely, in regions with gentle gradient changes, the calculated side length will approach the baseline size constant, resulting in larger mesh cells. To ensure algorithm robustness, the calculation results also need to undergo boundary checks to ensure that the final side length will not be less than a preset minimum size. In this embodiment, this minimum size is preferably set to 1.0 voxel unit length to prevent the generation of degenerate mesh cells. By repeating this process for each boundary voxel, the adaptive generation of the mesh size for the entire lesion surface can be achieved.

[0135] To further clarify the above process, a complete example is provided for calculating the side length of the corresponding mesh cell for a single target voxel. Before the calculation in this example begins, the system needs to pre-set key parameters. The baseline size constant is set to 5.0 to define the maximum side length of the mesh cell; the fixed constant serves as a sensitivity adjustment factor, preferably 0.01; and the minimum size set to ensure mesh quality is 1.0 voxel unit length.

[0136] In the calculation process, step S321 is first executed, where the system selects a target voxel located at the lesion boundary and successfully acquires the gradient magnitude within its 3×3×3 neighborhood. For simplicity, a representative set of values ​​is 15, 20, 18, 90, 110, 95, 22, 17, and 25, indicating the existence of a moderate-intensity boundary in this local region. Next, in step S322, the system calculates the statistical characteristics based on this set of gradient magnitudes. The calculated average value is approximately 45.78, and the standard deviation is approximately 39.85.

[0137] In step S323, the system uses the aforementioned statistics to determine the final grid cell edge length. First, the calculated mean and standard deviation are added together to obtain a sum of 85.63. Then, this sum is multiplied by a preset fixed constant of 0.01 to obtain a proportion representing local complexity of 0.8563. Finally, this proportion is subtracted from the baseline size constant of 5.0 to obtain a preliminary edge length of 4.1437. Before determining the final size, a boundary check is performed. The calculated preliminary size of 4.1437 is compared with the preset minimum size of 1.0; the result shows that the former is larger than the latter, therefore the calculation result is valid.

[0138] In summary, at this target voxel, the final side length of the generated tetrahedral mesh element was determined to be 4.1437 voxel units. This result clearly demonstrates the effect of the adaptive mechanism: due to the local complexity reflected by the gradient changes in this region, the generated mesh size was appropriately reduced from the maximum of 5.0, thus achieving a more refined modeling than the default size. Correspondingly, if the calculation occurs in a flat interior region where the gradient value is generally close to 0, the final calculated side length will be very close to the baseline size constant of 5.0.

[0139] See Figure 2 In step S40, the initial three-dimensional geometric surface is converted into an implicit function representation, and iteratively optimized using physics-driven surface evolution constraints to obtain an optimized three-dimensional surface model.

[0140] Step S41: Establish an axial bounding box that completely contains the initial three-dimensional geometric surface, and regularly divide the internal space of the bounding box into a three-dimensional voxel mesh. Calculate the shortest directed distance from the center of each voxel to the initial three-dimensional geometric surface, and generate an initial directed distance field as an implicit function.

[0141] Step S42: Construct an energy functional consisting of a linear combination of the integral term of the square of the mean curvature of the surface and the penalty term for the change in surface area;

[0142] Step S43: Minimize the energy functional using the numerical gradient descent method. In each iteration, update the distance values ​​of each voxel based on the spatial gradient of the current directed distance field and the variational derivative of the energy functional.

[0143] Step S44: When the change in the value of the energy functional between two consecutive iterations is less than one ten-thousandth of its value before the first iteration, the iteration is stopped.

[0144] Step S45: Extract the zero-value isosurface from the final directed distance field and convert it into a triangular mesh to generate the optimized three-dimensional surface model.

[0145] In step S40 of this invention, the initial three-dimensional geometric surface generated in the preceding steps is optimized to eliminate potential geometric noise, staircase effects, and irregular surface fluctuations, thereby generating a smoother, more natural, and physically consistent optimized three-dimensional surface model. The core idea of ​​this optimization process is to convert the explicit geometric surface into a mathematical expression called an implicit function, and to iteratively improve the surface quality by simulating a physically driven surface evolution process. This method not only efficiently handles complex topologies but also ensures the high smoothness of the final surface. This process is specifically executed sequentially through steps S41 to S45, transforming an imperfect initial model into a high-quality final result.

[0146] In the initial stage of the optimization process, namely step S41, a computational environment needs to be constructed for the initial 3D geometric surface. Specifically, a 3D axial bounding box is established that completely contains the initial surface. To ensure sufficient space for subsequent evolution, the size of this bounding box is typically expanded outward along the X, Y, and Z axes by a certain safety margin, for example, by 5% or a fixed 5-10 voxel units, based on the maximum boundary range of the initial surface. Subsequently, the space inside this bounding box is regularly discretized into a uniform 3D voxel mesh. The mesh resolution, i.e., the voxel size, is a key parameter affecting the final model accuracy and computational efficiency. In this embodiment, it can be set according to the slice thickness of the input CT image, for example, set to 0.5mm × 0.5mm × 0.5mm cubes.

[0147] After the mesh is constructed, the shortest directed distance from the center point of each voxel to the initial 3D geometric surface is calculated, thereby generating an initial directed distance field as an implicit function. The directed nature here means that the distance value can be positive, negative, or zero. In this embodiment, the voxel center points located outside the initial surface have a positive distance value; those located inside have a negative distance value; and points exactly on the surface have a zero distance value. To efficiently calculate this distance field, algorithms such as fast traversal or fast scan can be used instead of traversal searches, thus significantly improving computation speed while maintaining accuracy. Thus, the explicit triangular mesh surface is successfully converted into a directed distance field stored in the 3D mesh. The surface itself is defined as The zero isosurface.

[0148] Next, in step S42, a composite energy functional is constructed to guide the surface towards a more optimized morphological evolution. This energy functional is a quantitative assessment of the current geometric quality of the surface; the smaller the value, the higher the surface quality. In this embodiment, this energy functional... It consists of two linearly combined parts: one part is a term characterizing the bending energy of the surface, and the other part is a penalty term controlling the change in surface area. The specific mathematical expression is as follows: The first term is the average curvature of the surface. The integral of the square of the integral over the entire surface. Minimizing this integral term is equivalent to minimizing the total bending energy of the surface, which physically corresponds to the process of an elastic thin plate tending towards its flattest state. Geometrically, this strongly smooths the surface, eliminating high-frequency noise and sharp features. Mean curvature It can itself be accessed through a directed distance field Calculated, i.e. .

[0149] The second item is the total surface area. The penalty term is determined by a weighting coefficient. Adjustments are made. This function constrains the overall deformation of the surface during the smoothing process; for example, when… When it is positive, it penalizes excessive area growth, preventing the surface from expanding unreasonably; when... When the value is negative (more common in this case), it slightly drives the surface to shrink, a process similar to the effect of surface tension, which helps eliminate unwanted small bumps or voids, making the shape more compact. Weighting coefficient The value of needs to be set empirically based on the specific application scenario and the desired smoothing strength, for example, -0.01 can be used.

[0150] In step S43, the energy functional defined in step S42 is minimized using the numerical gradient descent method, which constitutes the core iterative process of surface evolution. The idea behind the gradient descent method is to guide the surface (i.e., the zero isosurface) along a path that minimizes the total energy. Move in the direction of the fastest descent. Within the implicit function framework, this is equivalent to updating the directed distance field of every voxel in the entire 3D voxel mesh. In each iteration, the new distance value for each voxel is based on the current directed distance field. and its spatial gradient and energy functionals about The update process is calculated using the variational derivative (i.e., the Euler-Lagrange equation). This update process can be represented as a discretized solution to a partial differential equation: .in, The time step must be small enough to ensure numerical stability (satisfying the CFL condition). It is the magnitude of the gradient vector at the current point, ensuring the consistency of the evolution rate in space; while The term, acting as the driving force for evolution, represents the negative variational derivative of the energy functional. For the energy functional in this embodiment, The term is a fourth-order differential operator related to the Laplacian operator for surfaces, which diffuses energy from high-curvature regions to low-curvature regions, thereby achieving global smoothing. In numerical implementation, The derivatives and higher-order derivatives are approximated on the voxel grid using finite difference schemes such as central difference.

[0151] To determine whether the surface evolution has reached a stable state, the entire energy functional is recalculated after each iteration. The value. When two consecutive iterations (e.g., the first iteration) Second and third The energy value calculated (time) and The change between them, i.e. It has become very small, specifically, less than or equal to the initial value of the energy functional before the first iteration. One ten-thousandth (i.e., satisfying) When the energy converges and the surface shape reaches a locally optimal stable state, the iteration stops. Furthermore, to prevent infinite loops caused by improper parameter settings or other reasons, this invention can also set a maximum iteration limit, such as 500 times, as a safety measure to forcibly terminate the iteration.

[0152] In step S45, after the iteration process terminates, a fully optimized and stable directed distance field is obtained. At this point, this implicit mathematical expression needs to be converted back into an explicit 3D geometric model usable for subsequent applications (such as visualization, 3D printing, or mechanical analysis). This conversion is accomplished by extracting zero-value isosurfaces from the final directed distance field. In this embodiment, the industry-standard moving cube algorithm is preferably used to achieve this. This algorithm examines each small cube in the 3D voxel mesh one by one, determining whether and how the zero-value isosurface passes through the cube by analyzing the sign combination of the directed distance values ​​at its eight vertices. Based on a preset triangulation lookup table, one or more small triangular patches are generated within each traversed cube to accurately approximate the local isosurface. The triangular patches generated from all voxel elements are combined to form a closed and continuous triangular mesh. This finally generated triangular mesh is the optimized 3D surface model described in this invention; it has a smooth surface, natural details, faithfully reflects the macroscopic morphology of the lesion, and eliminates high-frequency noise from the original data.

[0153] To further illustrate the construction scheme of the axial bounding box, in this embodiment, the method for establishing an axial bounding box that completely contains the initial three-dimensional geometric surface is as follows:

[0154] Step S411: Determine the position extrema of the initial three-dimensional geometric surface in the directions of three mutually orthogonal coordinate axes based on the spatial point cloud of the initial three-dimensional geometric surface, so as to define the initial boundary cubic region that just accommodates the surface.

[0155] Step S412: Calculate the safety margin for each dimension of the initial boundary cubic region. The safety margin consists of the relative expansion amount obtained by multiplying the initial span of the dimension by a preset scaling factor, plus a preset fixed spatial unit expansion amount.

[0156] Step S413: The sum of the relative expansion amount and the absolute expansion amount is taken as the total expansion amount of the corresponding dimension, and the boundary value of the dimension is updated accordingly. That is, the original boundary value is subtracted from the total expansion amount in the minimum value direction, and the original boundary value is added to the total expansion amount in the maximum value direction.

[0157] Step S414: Based on the updated boundary values ​​of all dimensions, the spatial range of the axial bounding box is finally determined.

[0158] In this embodiment, the specific implementation method for establishing an axial bounding box that completely contains the initial three-dimensional geometric surface is as follows: Based on the spatial point cloud of the initial three-dimensional geometric surface, traverse the three-dimensional coordinates of all its vertices, and determine the minimum and maximum values ​​in the three mutually orthogonal coordinate axes of the X, Y, and Z axes. For example, let the minimum value of the X coordinate of all vertices be... The maximum value is The minimum value of the Y-coordinate is The maximum value is The minimum value of the Z-coordinate is The maximum value is Therefore, with point ( , , ) is the smallest corner point, with point ( , , If the maximum corner point is , then an initial boundary cubic region that can just accommodate the initial three-dimensional geometric surface can be defined.

[0159] Subsequently, a safety margin is calculated for each dimension of the initial boundary cubic region. This safety margin consists of two parts: the first part is the relative expansion based on the initial span of that dimension, and the second part is a preset fixed spatial unit expansion. Specifically, for the X-axis dimension, its initial span is ( Multiplying this span by a preset scaling factor (for example, the factor can be set to 0.05) yields the relative expansion amount. At the same time, a fixed spatial expansion unit is set. (For example, corresponding to 5 voxel units in image space). Then the total expansion of this dimension (X-axis) is... The sum of the two: Repeat this calculation process for the Y-axis and Z-axis dimensions to obtain the total expansion. and The scaling factor and fixed spatial unit expansion amount can be set based on empirical values ​​of the resolution of the processed image, the size range of the lesion, and the space required for subsequent surface evolution.

[0160] Next, the boundary values ​​for each dimension are updated using the calculated total expansion. For the X-axis, the original minimum value is... Updated to , to the original maximum value Updated to Perform the same operation on the Y-axis and Z-axis, updating the results accordingly. This operation essentially involves uniformly expanding the initial boundary cubic region outward along each axis by the corresponding total expansion amount.

[0161] Finally, based on the updated boundary values ​​of all dimensions, the spatial extent of the axial bounding box is determined. The minimum corner coordinates of this bounding box are ( , , The coordinates of the largest corner point are ( , , The axial bounding box thus defined completely contains the initial three-dimensional geometric surface and provides the necessary space margin for its subsequent surface evolution process under implicit function representation, ensuring that the surface deformation during the evolution process has sufficient computational space and avoiding the optimization effect due to excessively tight boundary constraints.

[0162] See Figure 4 In step S50, the optimized three-dimensional surface model is back-projected onto the feature space of the second segmentation sub-network, and an attention weight map is generated based on the spatial inconsistency.

[0163] In this embodiment, step S50 includes:

[0164] Step S51: Obtain the target 3D feature map output by the second segmentation sub-network when processing the enhanced feature volume data;

[0165] Step S52: Transform the vertex coordinates of the optimized three-dimensional surface model to the spatial coordinate system corresponding to the target three-dimensional feature map;

[0166] Step S53: On the target 3D feature map, find the voxel position closest to the coordinates of each transformed vertex, and extract the multi-channel feature vector at that voxel position.

[0167] Step S54: Calculate the feature response value corresponding to each vertex based on the multi-channel feature vector;

[0168] Step S55: Calculate the absolute value of the difference between the value of the binarized three-dimensional lesion mask at the same vertex position and the feature response value, and use it as a measure of spatial inconsistency at that vertex.

[0169] Step S56: Based on the spatial inconsistency measure at all vertices, an attention weight map with the same spatial size as the target three-dimensional feature map is generated by three-dimensional interpolation.

[0170] In step S50 of this invention, a feedback mechanism is established from the geometric optimization result to the feature space of the neural network. Its core purpose is to locate and quantify the spatial inconsistency between the initial prediction result of the second segmentation sub-network and the ideal geometric shape after physical-driven optimization, and to encode this inconsistency into an attention weight map to guide subsequent iterative fine-tuning of network parameters. This step is achieved by back-projecting the optimized 3D surface model obtained in step S40 into the deep feature space of the second segmentation sub-network. Specifically, this process, through a series of operations in steps S51 to S56, transforms the geometric correction information into spatially targeted supervisory signals that can be directly used within the neural network.

[0171] Step S51 involves acquiring the target 3D feature map output by the second segmentation sub-network when processing the enhanced feature volume data. According to the cascaded segmentation architecture of the present invention, the second segmentation sub-network is responsible for fine-grained segmentation, and its input is enhanced feature volume data that incorporates the initial segmentation results of the first segmentation sub-network. The present invention does not compare the final binarized mask output by the second segmentation sub-network, but instead selects a deeper intermediate decoder feature map, generated before the final segmentation result, as the target. This target 3D feature map retains extremely rich multi-channel semantic information. The feature vector at each voxel location contains the network's complex judgment criteria for whether the point belongs to a lesion, background, or boundary, after combining the original image and preliminary localization information. Compared to the final binary result (either 0 or 1), its information content is much higher and better reflects the cognitive state within the network. This feature map is captured and temporarily stored when the second segmentation sub-network performs forward propagation on the enhanced feature volume data to generate the initial binarized mask.

[0172] Step S52 performs a crucial coordinate system alignment operation. Due to downsampling operations such as convolution and pooling in the neural network, the spatial size of the target 3D feature map is typically much smaller than the original CT image, for example, it might be 1 / 8 or 1 / 16 of the original size. However, the optimized 3D surface model generated in step S40 defines its vertex coordinates in the physical space coordinate system of the original CT image. Therefore, all vertex coordinates of the optimized 3D surface model must be transformed from the original coordinate system to the spatial coordinate system corresponding to the target 3D feature map. This transformation is typically a linear scaling operation, with the specific scaling factor determined by the total downsampling rate of the second segmentation sub-network from its input to the layer containing the target feature map. For example, if the network performs three downsampling steps of size 2 from the input to the layer, the total downsampling rate is 8, and the three components of the vertex coordinates need to be divided by 8 accordingly.

[0173] Step S53 involves extracting information from the coordinates of each vertex after the transformation in step S52 on the target 3D feature map. In the feature map space of the transformed vertex, the nearest neighbor voxel is found using the nearest neighbor sampling method. Once this nearest neighbor voxel is located, the complete multi-channel feature vector stored at that voxel location is extracted. For example, if the target feature map has 64 channels, a 64-dimensional vector is extracted. By repeating this operation for all vertices, a deep feature description from within the second segmentation sub-network is associated with each point on the optimized surface.

[0174] Step S54 interprets the multi-channel feature vector of each vertex extracted in the previous step and calculates the single feature response value corresponding to each vertex. Since the feature vector is high-dimensional and abstract, it needs to be converted into a scalar value directly related to the segmentation task, such as a probability value between 0 and 1. In this embodiment, a preferred implementation is to pass the extracted multi-channel feature vector through the last classifier layer in the second segmentation sub-network used to generate the final segmentation mask. This classifier is typically a 1×1×1 convolutional layer followed by a sigmoid activation function. Inputting the feature vector of each vertex into this classifier, the single output value can be regarded as the confidence or probability that the network determines the vertex location belongs to a lesion based on its deep features, i.e., the feature response value.

[0175] Step S55 calculates spatial inconsistency. This inconsistency is measured by comparing judgments from two different sources. The first source is the initial binary 3D lesion mask generated by the second segmentation sub-network in step S20, which represents the network's initial fine segmentation result without any geometric prior constraints. The second source is the feature response value at the vertex position of the optimization surface calculated in the previous step. Specifically, for each vertex on the optimization surface, its value at the corresponding position in the initial binary 3D lesion mask is first obtained; this value is typically 0 or 1. Then, the absolute value of the difference between this value and the feature response value of that vertex is calculated. This absolute difference is defined as the spatial inconsistency measure at that vertex. A high inconsistency measure, for example, close to 1, means that the deep features of the neural network exhibit extremely low lesion responses at the ideal lesion boundary defined by the geometric optimization model, or conversely, it indicates a significant cognitive conflict at this point, which is an area that the network needs to focus on learning and correcting in subsequent iterations.

[0176] Step S56 expands these sparsely distributed metrics on the surface vertices into a dense attention weight map with the same spatial dimensions as the target 3D feature map, based on the spatial inconsistency metrics calculated at all vertices. This process is implemented using a 3D interpolation method, specifically trilinear interpolation in this embodiment. Specifically, all vertices and their corresponding inconsistency metrics are distributed as known data points in the grid space of the 3D feature map. For any voxel with an unknown value in the grid, the eight nearest known vertex data are found, and a weighted average is calculated based on the 3D spatial distance from the voxel to these eight vertices to calculate the interpolation value for that voxel. After performing this operation on all voxels, a complete 3D attention weight map is generated. The numerical distribution in this map intuitively reflects the intensity and spatial location of the model-prediction inconsistency in the entire feature space. The highlighted areas are the regions where the network needs to apply higher attention in the next training round, thereby achieving precise guidance and correction of the segmentation boundaries.

[0177] Step S60: Feed the attention weight map back to the second segmentation sub-network, iteratively update the network parameters and make the segmentation boundary converge, and output the final segmentation result;

[0178] In step S60 of this invention, a closed-loop feedback and iterative optimization training process is performed. This process feeds the attention weight map generated in step S50 back to the second segmentation sub-network, iteratively updating the network parameters to cause the segmentation boundary to converge to a geometrically more reasonable position, and finally outputs a refined final segmentation result. This step is a key link in realizing the collaborative optimization of segmentation and reconstruction described in this invention.

[0179] The specific implementation of this feedback mechanism involves integrating the attention weight map into the design of its loss function during the training or fine-tuning phase of the second segmentation sub-network, thereby constructing a spatially weighted loss function. Before each iteration of training, the attention weight map generated in step S50, which has the same spatial size as the target 3D feature map, needs to be restored to the same spatial size as the probability map output by the second segmentation sub-network (i.e., the original input image) through upsampling methods such as trilinear interpolation. This operation ensures that each weight value in the attention weight map corresponds one-to-one with each voxel position in the network's output probability map.

[0180] During training iterations, the second segmentation sub-network receives the enhanced feature volume data generated in step S20 as input and performs a complete forward propagation to generate a three-dimensional probability map, where the value of each voxel represents the probability that it belongs to a lesion. The difference between this probability map and the corresponding true segmentation mask is quantified by a spatially weighted loss function. In this embodiment, a weighted binary cross-entropy loss can be used. For any voxel in the three-dimensional probability map, its contribution to the total loss is no longer equal, but modulated by the value in the upsampled attention weight map corresponding to that position. Specifically, the standard binary cross-entropy loss value at that voxel position is multiplied by its corresponding attention weight value. Since the attention weight value represents the spatial inconsistencies identified in previous steps, a higher weight value means that the position is a focal region where the network prediction conflicts with the geometric optimization model. Therefore, through this weighting method, the network disproportionately amplifies the errors made in these inconsistent regions when calculating the total loss, thereby generating a larger loss gradient.

[0181] After the loss value is calculated, the standard backpropagation algorithm is executed. This algorithm calculates the gradient of the loss with respect to all trainable parameters (including convolutional kernel weights and bias terms) in the second segmentation sub-network based on the calculated weighted loss. These gradients explicitly point in the direction that minimizes segmentation errors in high-interest regions. A numerical optimizer, preferably the Adam optimizer in this embodiment, makes small updates to the parameters of the second segmentation sub-network based on these gradients. Through this complete forward propagation, weighted loss calculation, backpropagation, and parameter update, the second segmentation sub-network completes a self-correction guided by geometric inconsistencies.

[0182] This iterative update process is repeated until the segmentation boundary reaches a convergent state. Convergence can be determined in several ways. A preferred method is to monitor the stability of the segmentation results between consecutive iterations. Specifically, after each iteration, a temporary binary segmentation mask is generated, and its Dessian similarity coefficient with the mask generated in the previous iteration is calculated. When this coefficient is consistently higher than a preset, extremely high threshold (e.g., 0.999) for several consecutive iterations (e.g., 3), the segmentation boundary is considered to no longer change significantly, and the network has converged. Alternatively, the weighted loss value can be monitored; if its decrease is less than a minimum value over several consecutive iterations, convergence can also be considered. Furthermore, to ensure the algorithm terminates, a maximum number of iterations can be set, such as 20. Once any convergence condition is met, the iterative update process terminates.

[0183] After the iteration process terminates, the resulting second segmentation sub-network is a final network model refined with geometric information. To generate the final output of this step, this converged network model is applied to the original enhanced feature volume data, performing a final forward propagation to obtain the final 3D probability map. A fixed threshold (e.g., 0.5) is applied to this probability map for binarization, generating the final segmentation result. This result is a binarized 3D lesion mask that is accurate at the voxel level, smooth in macroscopic morphology, and topologically correct, providing high-quality input for the final 3D reconstruction in subsequent step S70.

[0184] See Figure 4 In step S70, three-dimensional reconstruction is performed again based on the final segmentation result, and surface optimization is performed using geometric consistency constraints derived from network features to generate the final three-dimensional reconstruction model.

[0185] In this embodiment, step S70 includes:

[0186] Step S71: Based on the final segmentation result, extract the corresponding lesion voxel set from the unified three-dimensional volume data, and perform mesh discretization and vertex optimization to construct a secondary initial three-dimensional geometric surface;

[0187] Step S72: Convert the secondary initial three-dimensional geometric surface into an implicit function representation;

[0188] Step S73: Obtain the target 3D feature map generated by the second segmentation sub-network during the output of the final segmentation result;

[0189] Step S74: Based on the target 3D feature map, calculate the distribution divergence of the multi-channel feature vectors at each voxel location, and construct a geometric consistency term to constrain the surface evolution.

[0190] Step S741: For each voxel position in the target 3D feature map, extract the multi-channel feature vector of all voxels in the 3D neighborhood centered on that voxel position.

[0191] Step S742: Calculate the covariance matrix of all multi-channel feature vectors in the three-dimensional neighborhood;

[0192] Step S743: Solve for the eigenvalues ​​of the covariance matrix, and use the sum of the eigenvalues ​​as the characteristic distribution divergence value of the central voxel position;

[0193] Step S744: Based on the feature distribution divergence values ​​at all voxel locations, map the feature distribution divergence values ​​to the corresponding geometric constraint weights using a preset monotonically decreasing function.

[0194] Step S745: Using the geometric constraint weights, construct the geometric consistency term, which is the integral of the geometric constraint weights and the square of the implicit function gradient magnitude over the surface domain.

[0195] Step S75: The geometric consistency term, curvature smoothing term, and area constraint term are weighted and combined to form a new energy functional.

[0196] Step S76: Minimize the new energy functional using the numerical gradient descent method to optimize the implicit function;

[0197] Step S77: Extract the zero-value isosurface from the optimized implicit function and convert it into a triangular mesh to generate the final three-dimensional reconstruction model.

[0198] In step S70, this invention performs the final 3D reconstruction, aiming to generate a final 3D reconstruction model that combines high geometric smoothness and semantic fidelity, faithfully reflecting the features of the original image. The core technological innovation of this step lies in the fact that it does not simply convert the final segmentation mask into a 3D model, but rather introduces geometric consistency constraints derived from the depth features of the segmentation network to intelligently guide the surface optimization process, thereby achieving accurate preservation of lesion morphology and adaptive filtering of image noise. This reconstruction process is based on... Figure 5 As shown, the steps S71 to S77 are implemented sequentially.

[0199] In step S71, this method first constructs a high-quality secondary initial 3D geometric surface based on the final and most accurate segmentation result obtained in step S60. Specifically, based on this final segmentation result, the system accurately extracts all voxels identified as lesion regions from the unified 3D volume data generated in step S10, thus forming an updated lesion voxel set. Subsequently, the system fully replicates the adaptive mesh discretization and image gradient-based vertex optimization process described in step S30. That is, adaptive tetrahedral meshing is applied to the updated lesion voxel set, generating finer mesh units in areas with complex boundary morphology. Simultaneously, using the grayscale gradient information within the unified 3D volume data, the surface vertices of the initial mesh are precisely snapped to the positions in the image with the most intense gradient response. Through this process, a secondary initial 3D geometric surface is generated as the starting point for the final reconstruction. Because this surface originates from a more accurate segmentation mask, its initial morphology and topology highly approximate the real lesion, laying a solid foundation for subsequent semantic-guided optimization.

[0200] In step S72, the quadratic initial three-dimensional geometric surface constructed in step S71 is converted into an implicit function representation to facilitate subsequent surface evolution based on the level set method. This conversion process is similar to step S41, achieved by calculating the shortest directed distance from the center of each voxel to the surface on a three-dimensional voxel mesh that completely covers the surface, thereby generating a three-dimensional directed distance field. This directed distance field implicitly defines the explicit triangular mesh surface as its zero-value isosurface. This representation method has inherent technical advantages for handling complex geometric deformations and topological changes, as well as implementing evolutionary solutions based on partial differential equations.

[0201] In step S73, to obtain deep semantic information to guide surface optimization, the system needs to extract the target 3D feature map produced by the second segmentation sub-network during the output of the final segmentation result. This target 3D feature map is preferably from a deep level in the decoder path of the second segmentation sub-network; a specific example is the last or penultimate layer immediately before the upsampling operation in the U-Net architecture. The feature map at this level has a spatial resolution lower than the original image. The multi-channel feature vector corresponding to each voxel contains a deep understanding and highly abstract representation of the local context, texture, and tissue type of the deep neural network at that location, rather than merely the original image grayscale values. These deep features are crucial for accurately distinguishing the true anatomical boundaries of lesions from false boundaries caused by noise or artifacts.

[0202] In the core stage of the final 3D reconstruction, namely step S74, this invention constructs a geometric consistency term driven by deep network features to apply semantically aware spatial adaptive constraints during surface optimization. The implementation method of this process is as follows:

[0203] The process begins in step S741, where the system extracts feature information from the local region of each voxel position in the target 3D feature map obtained in the preceding step S73. Specifically, a 3D neighborhood is defined centered on the currently processed target voxel position. In this embodiment, this neighborhood is preferably a cube region with a size of 3×3×3 voxels. Subsequently, the system extracts the corresponding multi-channel feature vectors from all 27 voxel positions (including the central voxel) contained within this cube neighborhood. These feature vectors are a highly abstract representation of the local image content by a deep neural network, and their dimension C is equal to the number of channels in the target 3D feature map. Through this operation, for each spatial point on the feature map, a set of 27 C-dimensional feature vectors can be obtained, which comprehensively describes the local semantic environment around the point.

[0204] Next, in step S742, the covariance matrix is ​​calculated for the set of local feature vectors extracted for each voxel location in the previous step. Assuming this set contains N C-dimensional feature vectors, the calculated covariance matrix will be a positive semi-definite matrix of dimension C×C. The diagonal elements of this matrix represent the variance of each feature channel within its local neighborhood, while the off-diagonal elements represent the covariance between different feature channels. Therefore, this covariance matrix fully quantifies the distribution pattern and internal correlation structure of multi-channel feature vectors within the local neighborhood, providing a solid statistical foundation for subsequent divergence calculations.

[0205] In step S743, the present invention solves for all C real eigenvalues ​​of the C×C dimensional covariance matrix calculated in the previous step. These eigenvalues ​​represent the variance of the eigenvector data along the principal component directions. Subsequently, these C eigenvalues ​​are summed, and the sum is used as the feature distribution divergence value defined at the central voxel location. From a technical perspective, the sum of these eigenvalues ​​is equivalent to the trace of the covariance matrix, which quantifies the overall variance of the local eigenvector set in the feature space. A large divergence value means that the eigenvectors in the neighborhood are widely and discretely distributed, which usually corresponds to regions with high semantic heterogeneity in the image, such as tissue boundaries, complex textures, or lesion edges. Conversely, a small divergence value indicates that the eigenvectors in the neighborhood are highly clustered, corresponding to regions with high semantic homogeneity, such as the interior of lesions or normal tissue. By repeating this calculation for all voxel locations in the target 3D feature map, a 3D divergence quantification field with the same size as the feature map space can be generated.

[0206] Subsequently, in step S744, the system needs to map the previously calculated feature distribution divergence values ​​to geometric constraint weights for surface evolution. To achieve semantically guided adaptive smoothing, this mapping relationship is designed as a monotonically decreasing function; that is, regions with larger feature divergences should receive smaller constraint weights to preserve details, while regions with smaller feature divergences should receive larger constraint weights to apply stronger smoothing. In this embodiment, this mapping is implemented using a function of the form w = 1 / (1 + α × S), where S is the feature distribution divergence value at the voxel location, α is a preset positive real hyperparameter used to adjust the sensitivity of the weights to changes in the divergence value, and w is the finally calculated geometric constraint weight, with a value ranging from 0 to 1. Applying this mapping function to the entire 3D divergence map scalar field generates a 3D geometric constraint weight map.

[0207] Finally, in step S745, the geometric consistency term for the final energy functional is constructed using the geometric constraint weight graph generated in the previous step. Mathematically, this geometric consistency term is defined as an integral over the current surface domain S, with the geometric constraint weights as the integrand. The square of the magnitude of the implicit function gradient is... The product of these terms. The introduction of this term allows the subsequent energy minimization process to adaptively adjust based on deep semantic information: in the geometric constraint weights... In homogeneous regions with large values, this term significantly penalizes the surface gradient, strongly driving the surface to become flat, thus achieving effective smoothing and noise reduction; however, in geometric constraint weights... In boundary regions with smaller values, the penalty effect of this term is weakened, allowing the surface to maintain a larger gradient, thereby effectively protecting the sharp features and geometric details of the true lesion boundaries identified by the deep segmentation network. Thus, a geometric consistency term capable of semantically guided surface optimization is constructed, providing a crucial mathematical constraint for generating a high-fidelity final 3D model.

[0208] In step S75, the geometric consistency term constructed in the previous step, a traditional curvature smoothing term (which can be the integral of the square of the average curvature of the surface), and an area constraint term (used to control the expansion or contraction of the surface during evolution) are combined linearly in a weighted manner. This combination constitutes a composite energy functional containing deep semantic information. This composite energy functional can synergistically balance the requirements of geometric smoothing and semantic fidelity. Specifically, in homogeneous regions with small feature divergence values, the corresponding geometric constraint weights are larger, and the energy functional will impose a strong penalty on non-zero gradients, thereby prompting the surface to be smoothed extensively. Conversely, in boundary regions with large feature divergence values, the geometric constraint weights are smaller, and the penalty of the energy functional on the gradient is weakened, thus allowing the surface to maintain its inherent sharp shape, thereby effectively protecting the details of the true lesion boundaries identified by the deep neural network.

[0209] In step S76, the numerical gradient descent method is used to minimize the composite energy functional defined in step S75 to optimize the implicit function generated in step S72. This optimization process manifests as a level set evolution process. Specifically, the system calculates the evolution velocity field based on the variational derivative of the energy functional and its corresponding Euler-Lagrange equation, and updates the entire three-dimensional directed distance field in each iteration step based on this velocity field. The evolution of the surface is thus guided by depth semantic information, smoothing is performed in areas requiring smoothness, and its shape is maintained in areas where details need to be preserved. This iterative process also has a convergence criterion; the evolution process terminates when the numerical rate of change of the energy functional is lower than a preset threshold, or when the number of iterations reaches a preset upper limit.

[0210] In step S77, after the surface evolution process converges, the system extracts the zero-value isosurface from the optimized stable directed distance field and converts it into a triangular mesh representation. This extraction process can be implemented using the classic Moving Cubes algorithm. The resulting triangular mesh constitutes the final 3D reconstruction model of this invention. This model not only has a smooth surface without significant geometric defects, but its morphological details are also highly consistent with the image semantics understood by deep neural networks, thereby achieving high-fidelity reconstruction of the lesion's 3D morphology.

[0211] Although the steps in the above embodiments are described in the above order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not need to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple variations are all within the protection scope of this invention.

[0212] See Figure 5 The second embodiment of the present invention provides an artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction system, based on an artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method. The system includes:

[0213] The image acquisition and fusion module is configured to acquire multimodal medical images, perform registration and fusion, and generate unified three-dimensional volume data;

[0214] The cascaded segmentation module is configured to input the unified 3D volume data into the first segmentation sub-network of the cascaded segmentation network to obtain a primary segmentation result; to perform feature concatenation between the primary segmentation result and the unified 3D volume data, and input the result into the second segmentation sub-network of the cascaded segmentation network to generate a binarized 3D lesion mask.

[0215] The three-dimensional surface initial construction module is configured to extract lesion voxels based on the binarized three-dimensional lesion mask and construct the initial three-dimensional geometric surface through adaptive mesh discretization and vertex optimization.

[0216] The surface optimization module is configured to convert the initial three-dimensional geometric surface into an implicit function representation and perform iterative optimization through physical-driven surface evolution constraints to obtain an optimized three-dimensional surface model.

[0217] The attention map generation module is configured to back-project the optimized 3D surface model to the feature space of the second segmentation sub-network and generate an attention weight map based on spatial inconsistency.

[0218] The feedback training module is configured to feed the attention weight map back to the second segmentation sub-network, iteratively update the network parameters and converge the segmentation boundary, and output the final segmentation result.

[0219] The final 3D reconstruction module is configured to perform 3D reconstruction again based on the final segmentation result, and perform surface optimization with geometric consistency constraints derived from network features to generate the final 3D reconstruction model.

[0220] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related explanations of the methods described above can be found in the corresponding processes in the foregoing system embodiments, and will not be repeated here.

[0221] It should be noted that the artificial intelligence-based medical image lesion segmentation and 3D reconstruction system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.

[0222] An electronic device according to a third embodiment of the present invention includes:

[0223] At least one processor;

[0224] and a memory communicatively connected to at least one of the processors;

[0225] The memory stores instructions that can be executed by the processor to implement the aforementioned artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method.

[0226] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions, which are executed by the computer to implement the above-described artificial intelligence-based medical image lesion segmentation and three-dimensional reconstruction method.

[0227] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the storage device and processing device described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0228] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0229] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0230] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence, characterized in that, The method includes: Acquire multimodal medical images and perform registration and fusion to generate unified three-dimensional volume data; The unified 3D volume data is input into the first segmentation sub-network of the cascaded segmentation network to obtain a primary segmentation result; the primary segmentation result is concatenated with the unified 3D volume data and input into the second segmentation sub-network of the cascaded segmentation network to generate a binary 3D lesion mask; wherein, both the first segmentation sub-network and the second segmentation sub-network are 3D U-Net networks; Based on the binarized 3D lesion mask, lesion voxels are extracted, and an initial 3D geometric surface is constructed through adaptive mesh discretization and vertex optimization. The adaptive mesh discretization refers to generating tetrahedral mesh units of adaptive size centered on the outer boundary voxels of the lesion voxel set. The adaptive size is dynamically adjusted based on the average and standard deviation of the gray-level gradient amplitudes of all voxels in the target voxel's spatial neighborhood. The vertex optimization refers to selecting the voxel position with the largest gray-level gradient amplitude from the 3D spatial neighborhood of the vertex for any vertex located in the boundary region of the binarized 3D lesion mask, and updating the spatial coordinates of the vertex to the selected voxel position. The initial three-dimensional geometric surface is converted into an implicit function representation and iteratively optimized through physical-driven surface evolution constraints to obtain an optimized three-dimensional surface model. The physical-driven surface evolution constraints refer to constructing an energy functional that includes the square integral term of the average curvature of the surface and a penalty term for the change in surface area, and minimizing the energy functional through the numerical gradient descent method to update the directed distance field represented by the implicit function. The optimized 3D surface model is back-projected into the feature space of the second segmentation sub-network, and an attention weight map is generated based on spatial inconsistency; wherein, spatial inconsistency refers to calculating the absolute value of the difference between the value of the binarized 3D lesion mask at the vertex position and the feature response value; The attention weight map is fed back to the second segmentation sub-network, the network parameters are iteratively updated and the segmentation boundary is converged, and the final segmentation result is output; wherein, the iterative update refers to integrating the attention weight map into the loss function of the second segmentation sub-network to construct a spatially weighted loss function, executing the backpropagation algorithm to update the parameters of the second segmentation sub-network, and monitoring the stability of the segmentation result between two consecutive iterations. When the change in the segmentation result is less than a preset threshold, the iteration is stopped. Based on the final segmentation result, three-dimensional reconstruction is performed again, and surface optimization is performed using geometric consistency constraints derived from network features to generate the final three-dimensional reconstruction model. The geometric consistency constraints derived from network features refer to calculating the distribution divergence of multi-channel feature vectors at each voxel position based on the target three-dimensional feature map output by the second segmentation sub-network, and constructing a geometric consistency term to constrain the surface evolution.

2. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 1, characterized in that, The method for acquiring, registering, and fusing multimodal medical images to generate unified three-dimensional volumetric data is as follows: At least two three-dimensional medical image sequences of the same subject are acquired, and each three-dimensional medical image sequence is preprocessed to unify spatial resolution and normalize grayscale intensity. The non-reference image sequence is downsampled to obtain a low-resolution image. The initial three-dimensional deformation field is estimated based on the low-resolution image. Then, the initial three-dimensional deformation field is iteratively optimized at the original resolution to obtain the final three-dimensional deformation field. Based on the final three-dimensional deformation field, nonlinear spatial registration is performed on each sequence, and multimodal voxel features are fused using a channel attention mechanism within the registration space to generate the unified three-dimensional volume data.

3. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 1, characterized in that, The method for feature concatenating the primary segmentation result with the unified 3D volume data is as follows: The initial segmentation result is processed into single-channel probabilistic volume data with the same spatial dimension as the unified three-dimensional volume data; The single-channel probability volume data is concatenated with the unified three-dimensional volume data along the channel dimension to form enhanced feature volume data; The enhanced feature volume data is fused by convolutional processing along the channel dimension to generate the input of the second segmentation sub-network.

4. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 1, characterized in that, Based on the aforementioned binary 3D lesion mask, lesion voxels are extracted, and an initial 3D geometric surface is constructed through adaptive mesh discretization and vertex optimization. The method is as follows: Extract the set of voxels marked as lesions by the binarized three-dimensional lesion mask from the unified three-dimensional volume data; Using the outer boundary voxel of the voxel set as the center, generate tetrahedral mesh elements of adaptive size to discretize the three-dimensional space inside the lesion; For each voxel in the voxel set, calculate the gray-level gradient direction and magnitude of all voxels in the three-dimensional spatial neighborhood of the voxel. For any vertex in the tetrahedral mesh cell located in the boundary region of the binary three-dimensional lesion mask, select the voxel position with the largest gray-scale gradient magnitude from the three-dimensional spatial neighborhood where the vertex is located, and update the spatial coordinates of the vertex to the selected voxel position. After traversing and optimizing the vertices of all boundary regions, the initial three-dimensional geometric surface is formed.

5. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 4, characterized in that, In generating tetrahedral mesh elements with adaptive dimensions, the method for calculating the adaptive dimensions is as follows: Obtain the grayscale gradient magnitude of the target voxel and all voxels in the spatial neighborhood of the target voxel; Calculate the average value and standard deviation of the gray-level gradient magnitude within the spatial neighborhood; Based on a preset reference size constant, the side length of the tetrahedral mesh cell to be generated at the target voxel is calculated by subtracting the ratio of the sum of the average value and the standard deviation from the reference size constant. The ratio is the product of the sum of the average and the standard deviation and a preset fixed constant.

6. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 1, characterized in that, The initial three-dimensional geometric surface is converted into an implicit function representation, and then iteratively optimized using physics-driven surface evolution constraints to obtain an optimized three-dimensional surface model. The method is as follows: An axial bounding box is constructed that completely contains the initial three-dimensional geometric surface, and the internal space of the bounding box is regularly divided into a three-dimensional voxel mesh. The shortest directed distance from the center of each voxel to the initial three-dimensional geometric surface is calculated, and an initial directed distance field is generated as an implicit function. Construct an energy functional consisting of an integral term of the square of the mean curvature of the surface and a penalty term for changes in the surface area; The energy functional is minimized using the numerical gradient descent method. In each iteration, the distance values ​​of each voxel are updated based on the spatial gradient of the current directed distance field and the variational derivative of the energy functional. The iteration stops when the change in the value of the energy functional between two consecutive iterations is less than one ten-thousandth of the value before the first iteration. Zero-value isosurfaces are extracted from the final directed distance field and converted into triangular meshes to generate the optimized 3D surface model.

7. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 3, characterized in that, The optimized 3D surface model is back-projected into the feature space of the second segmentation sub-network, and an attention weight map is generated based on spatial inconsistency. The method is as follows: Obtain the target 3D feature map output by the second segmentation sub-network when processing the enhanced feature volume data; The vertex coordinates of the optimized 3D surface model are transformed into the spatial coordinate system corresponding to the target 3D feature map; On the target 3D feature map, find the voxel position closest to the coordinates of each transformed vertex, and extract the multi-channel feature vector at that voxel position. Based on the multi-channel feature vector, calculate the feature response value corresponding to each vertex; The absolute value of the difference between the value of the binarized three-dimensional lesion mask at the same vertex position and the feature response value is calculated as a measure of spatial inconsistency at that vertex. Based on the spatial inconsistency measure at all vertices, an attention weight map with the same spatial size as the target 3D feature map is generated by 3D interpolation.

8. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 1, characterized in that, Based on the final segmentation result, 3D reconstruction is performed again, and surface optimization is performed using geometric consistency constraints derived from network features to generate the final 3D reconstruction model. The method is as follows: Based on the final segmentation result, the corresponding set of lesion voxels is extracted from the unified three-dimensional volume data, and mesh discretization and vertex optimization are performed to construct a secondary initial three-dimensional geometric surface. The quadratic initial three-dimensional geometric surface is converted into an implicit function representation; Obtain the target 3D feature map generated by the second segmentation sub-network during the output of the final segmentation result; Based on the target 3D feature map, calculate the distribution divergence of multi-channel feature vectors at each voxel position, and construct a geometric consistency term to constrain the surface evolution; The geometric consistency term is combined with the curvature smoothing term and the area constraint term in a weighted manner to form a new energy functional. The new energy functional is minimized using the numerical gradient descent method to optimize the implicit function; The zero-value isosurface is extracted from the optimized implicit function and converted into a triangular mesh to generate the final 3D reconstruction model.

9. The method for medical image lesion segmentation and three-dimensional reconstruction based on artificial intelligence according to claim 8, characterized in that, Based on the target 3D feature map, the distribution divergence of multi-channel feature vectors at each voxel location is calculated, and a geometric consistency term for constraining surface evolution is constructed. The method is as follows: For each voxel position in the target 3D feature map, extract the multi-channel feature vector of all voxels in the 3D neighborhood centered on that voxel position; Calculate the covariance matrix of all multi-channel feature vectors within the three-dimensional neighborhood; Solve for the eigenvalues ​​of the covariance matrix, and use the sum of the eigenvalues ​​as the feature distribution divergence value at the voxel location; Based on the feature distribution divergence values ​​at all voxel locations, the feature distribution divergence values ​​are mapped to the corresponding geometric constraint weights through a preset monotonically decreasing function; Using the geometric constraint weights, the geometric consistency term is constructed, which is the integral of the geometric constraint weights and the square of the implicit function gradient magnitude over the surface domain.

10. A medical image lesion segmentation and three-dimensional reconstruction system based on artificial intelligence, based on the medical image lesion segmentation and three-dimensional reconstruction method based on artificial intelligence as described in any one of claims 1-9, characterized in that, The system includes: The image acquisition and fusion module is configured to acquire multimodal medical images, perform registration and fusion, and generate unified three-dimensional volume data; The cascaded segmentation module is configured to input the unified 3D volume data into the first segmentation sub-network of the cascaded segmentation network to obtain a primary segmentation result; to perform feature concatenation between the primary segmentation result and the unified 3D volume data, and input the result into the second segmentation sub-network of the cascaded segmentation network to generate a binarized 3D lesion mask; wherein, both the first segmentation sub-network and the second segmentation sub-network are 3D U-Net networks. The initial 3D surface construction module is configured to extract lesion voxels based on the binarized 3D lesion mask and construct an initial 3D geometric surface through adaptive mesh discretization and vertex optimization. The adaptive mesh discretization refers to generating tetrahedral mesh units of adaptive size centered on the outer boundary voxels of the lesion voxel set. The adaptive size is dynamically adjusted based on the average and standard deviation of the gray-level gradient amplitudes of all voxels in the spatial neighborhood of the target voxel. The vertex optimization refers to selecting the voxel position with the largest gray-level gradient amplitude from the 3D spatial neighborhood of the vertex for any vertex located in the boundary region of the binarized 3D lesion mask, and updating the spatial coordinates of the vertex to the selected voxel position. The surface optimization module is configured to convert the initial three-dimensional geometric surface into an implicit function representation and perform iterative optimization through physical-driven surface evolution constraints to obtain an optimized three-dimensional surface model. The physical-driven surface evolution constraints refer to constructing an energy functional that includes the square integral term of the average curvature of the surface and a penalty term for the change in surface area, and minimizing the energy functional through the numerical gradient descent method to update the directed distance field represented by the implicit function. The attention map generation module is configured to back-project the optimized 3D surface model to the feature space of the second segmentation sub-network and generate an attention weight map based on spatial inconsistency; wherein, the spatial inconsistency refers to calculating the absolute value of the difference between the value of the binarized 3D lesion mask at the vertex position and the feature response value. The feedback training module is configured to feed the attention weight map back to the second segmentation sub-network, iteratively update the network parameters and converge the segmentation boundary, and output the final segmentation result; wherein, the iterative update refers to integrating the attention weight map into the loss function of the second segmentation sub-network to construct a spatially weighted loss function, executing the backpropagation algorithm to update the parameters of the second segmentation sub-network, and monitoring the stability of the segmentation result between two consecutive iterations, stopping the iteration when the change in the segmentation result is less than a preset threshold; The final 3D reconstruction module is configured to perform 3D reconstruction again based on the final segmentation result, and perform surface optimization with geometric consistency constraints derived from network features to generate the final 3D reconstruction model; wherein, the geometric consistency constraints derived from network features refer to calculating the distribution divergence of multi-channel feature vectors at each voxel position based on the target 3D feature map output by the second segmentation sub-network, and constructing a geometric consistency term to constrain the surface evolution.

Images