Lung image segmentation method based on improved SAM2

Abstract

The invention discloses a lung image segmentation method based on improved SAM2. The method comprises the following steps: obtaining a standardized lung CT image sequence; candidate focus area centers are extracted from the saliency heat map, and an automatic prompt point set is formed; generating a multi-category prompt graph; generating a global context feature map; generating a preliminary segmentation mask based on the nodule probability graph, the cavity probability graph and the fiber probability graph; generating a boundary enhancement mask based on a gradient consistency rule and an edge continuity rule; performing three-dimensional connectivity analysis on the focus morphological map to obtain a morphological map correction mask; and reconstructing a three-dimensional focus structure body set. The method overcomes the defects of boundary fuzziness and pathology category confusion of an existing method.

Description

Technical Field

[0001] This invention relates to the field of lung imaging technology, and more particularly to a lung image segmentation method based on an improved SAM2. Background Technology

[0002] With the continuous development of medical imaging equipment and computer vision technology, automatic analysis methods based on lung CT images have been widely applied in clinical scenarios such as lung nodule detection, lung lesion screening, and auxiliary diagnosis. Lung CT images can present the structural information of lung tissue at voxel-level resolution, providing a data foundation for the detailed analysis of lesions. However, in actual clinical practice, lung lesions often exhibit multiple and complex characteristics. The lungs of the same patient may simultaneously contain multiple lesion types such as nodules, cavities, and fibrosis. Different lesions are spatially adjacent, overlapping, or adhered to each other, posing a significant technical challenge to automatic segmentation.

[0003] Existing lung lesion segmentation methods are mainly based on convolutional neural networks or Transformer architectures. They learn the grayscale features of pixels or voxels to distinguish lesions from the background. While these methods are effective for single lesions or lesions with relatively regular shapes, they have significant limitations in complex scenarios with multiple lung lesions. Firstly, fibrous strands, cavity walls, and vascular structures have highly similar grayscale values ​​in CT images and are often spatially connected. Existing methods mainly rely on local intensity or shallow texture features, making it difficult to accurately distinguish morphologically similar but pathologically different tissues at the voxel level, easily leading to blurred boundaries, category confusion, and missegmentation. Secondly, when faced with spatial adhesion between lesions or fractures within the lesion structure itself, existing segmentation models typically only perform simple post-processing corrections, failing to guarantee the continuity and consistency of the segmentation results in three-dimensional space at the topological level.

[0004] In recent years, general segmentation models have gradually become a research hotspot. They are usually pre-trained on large-scale natural image data and have strong generalization ability. However, general segmentation models often rely on interactive prompts to achieve accurate segmentation, making it difficult to achieve fully automated application in medical imaging scenarios. Due to the significant differences between natural images and medical images in terms of imaging mechanisms, texture distribution, and structural scale, existing general segmentation models have limited response capabilities to small nodules, complex cavity boundaries, and slender fibrous structures in lung CT images, which can easily lead to problems such as missed lesions, fragmented segmentation results, or distorted topological structures. Summary of the Invention

[0005] One objective of this invention is to propose a lung image segmentation method based on an improved SAM2, which overcomes the shortcomings of existing methods such as blurred boundaries and confusion of pathological categories.

[0006] A lung image segmentation method based on an improved SAM2 according to an embodiment of the present invention includes:

[0007] S1. Acquire lung CT image sequences and preprocess them to obtain standardized lung CT image sequences;

[0008] S2. Input the standardized lung CT image sequence into the saliency detection module, generate a saliency heatmap based on Hounsfield Unit distribution, texture gradient information and spatial consistency rules, extract the center of candidate lesion region on the saliency heatmap and form an automatic prompting point set;

[0009] S3. Input the standardized lung CT image sequence and the set of auto-cue points into the medical cue encoder of the improved SAM2 model to generate multi-category cue maps;

[0010] S4. Input the standardized lung CT image sequence into the image encoder of the improved SAM2 model, and fuse the multi-class cue map into the encoder feature path to generate a global context feature map;

[0011] S5. Input the global context feature map into the multi-head semantic decoding module, output the nodule probability map, hole probability map and fiber probability map according to the nodule semantic head, hole semantic head and fiber semantic head respectively, and generate a preliminary segmentation mask based on the nodule probability map, hole probability map and fiber probability map;

[0012] S6. Input the preliminary segmentation mask, nodule probability map, hole probability map and fiber probability map into the boundary refinement module, and generate a boundary enhancement mask based on gradient consistency rules and edge continuity rules;

[0013] S7. Input the boundary enhancement mask into the linkage morphology reconstruction module, construct the lesion morphology map based on the nodule topology template, cavity topology template and fiber topology template, and perform three-dimensional connectivity analysis on the lesion morphology map to obtain the morphology map correction mask;

[0014] S8. Perform semantic stripping on the adhered regions of the morphological map correction mask and perform morphological connection on the broken regions to obtain the topological consistency optimized mask. Map the topological consistency optimized mask to the three-dimensional coordinate system and perform voxel-by-voxel fusion with the standardized lung CT image sequence to reconstruct the three-dimensional lesion structure set.

[0015] Optionally, the preprocessing includes noise suppression, lung field region extraction, and grayscale normalization.

[0016] Optionally, S2 includes:

[0017] S21. Based on the normalized membership function of the standardized HU value within the preset density range, extract the high-density response terms corresponding to nodules and fibrous tissues and the low-density response terms corresponding to cavities from the standardized lung CT image sequence, and then weight and fuse them to obtain the density distribution response map.

[0018] S22. Based on the gradient components of the standardized HU value in the x, y, and z directions of the three-dimensional coordinate system, calculate the three-dimensional gradient magnitude at each voxel coordinate, and use the three-dimensional gradient magnitude as the texture gradient response of the voxel to obtain the texture gradient response map.

[0019] S23. Using the coordinates of each voxel in the three-dimensional neighborhood of the voxel as a reference, sum the numerical differences between the density distribution response map and the texture gradient response map for each voxel to obtain the spatial consistency penalty term. Apply a negative exponential mapping to the spatial consistency penalty term based on the spatial consistency suppression coefficient to obtain the spatial consistency weight map.

[0020] S24. After normalizing the density distribution response map and texture gradient response map respectively, perform weighted summation and multiply it with the spatial consistency weight map to generate a significant heatmap;

[0021] S25. Apply a significance threshold to the significance heatmap, and aggregate voxels above the significance threshold into connected regions in three-dimensional space. The connected regions are used as candidate lesion regions.

[0022] S26. The weighted average of the three-dimensional coordinates is calculated by weighting the significance intensity of each voxel within the candidate lesion region, and the center of the candidate lesion region is taken as the center of the candidate lesion region. The centers of all candidate lesion regions form an automatic prompting point set.

[0023] Optionally, S3 includes:

[0024] S31. Input the standardized lung CT image sequence and the set of auto-cue points into the medical cue encoder of the improved SAM2 model. For each auto-cue point coordinate, the medical cue encoder selects voxels within a specified range to form a local voxel neighborhood centered on the auto-cue point coordinates.

[0025] S32. The medical prompt encoder obtains the mean HU and standard deviation HU based on the standardized HU values ​​of all voxels in the local voxel neighborhood;

[0026] S33. Based on the standardized HU value of each voxel in the local voxel neighborhood, calculate its gradient components in the x, y, and z directions respectively, and average the L2 norm of the gradient components of all voxels in the local voxel neighborhood to obtain the local gradient magnitude statistics.

[0027] S34. Calculate the structure tensor using the gradient vectors of voxels in the local voxel neighborhood;

[0028] S35. Using the structural tensor, calculate the morphological anisotropy index and the tubular uniformity index;

[0029] S36. The medical prompt encoder uses HU mean, HU standard deviation, local gradient magnitude statistics, morphological anisotropy index and tubular consistency index as input features to form a category prior vector;

[0030] S37. Based on the local gradient magnitude statistics and HU standard deviation, calculate the confidence weight of each automatic prompt point using a weighted normalization function;

[0031] S38. For each auto-suggestion point, generate a morphologically adaptive class-related kernel function according to the class type based on the feature vector information of the class prior vector, suggestion confidence weight, and structure tensor.

[0032] S39. The medical prompt encoder uses the prior weights of all automatic prompt points, the prompt confidence weights, and the class correlation kernel function to weight and superimpose the three types of responses of all voxels in a standardized lung CT image sequence to obtain a multi-class prompt heat volume.

[0033] S310. Generate a dense cue embedding tensor by passing the cue embedding mapping of the multi-category cue heat volume through the cue embedding map of the medical cue encoder;

[0034] The dense cue embedding tensor maintains a one-to-one correspondence with the standardized lung CT image sequence in space, and has a set number of cue embedding dimensions;

[0035] S311. The medical prompt encoder generates point location encoding vector, category prior encoding vector, and confidence encoding vector respectively based on the set of automatic prompt points, category prior vector, and prompt confidence weight, and then fuses them to output a sparse prompt embedding vector for each automatic prompt point;

[0036] S312. The medical cue encoder unifies the arrangement of dense cue embedding tensors and sparse cue embedding vectors to form a multi-category cue graph.

[0037] Optionally, S4 includes:

[0038] S41. Input the standardized lung CT image sequence into the image encoder of the improved SAM2 model to perform voxel block segmentation and feature aggregation to form an image voxel block embedding sequence;

[0039] S42. Based on the voxel block partitioning method, aggregate the dense cue embedding vectors of all voxel positions in each voxel block of the dense cue embedding tensor to form a voxel block-level dense cue embedding vector.

[0040] S43. For each feature layer of the image encoder, perform self-attention encoding on the output feature sequence of the previous layer to extract the self-attention features of the image itself;

[0041] S44. Construct a dual-source cue cross-attention mechanism, using the self-attention features of the image itself as the query, and the voxel block-level dense cue embedding sequence and sparse cue embedding vector set as the key and value, respectively calculate the attention response of image features to dense cues and sparse cues, and aggregate them to obtain cue fusion features;

[0042] S45. Construct an adaptive gating mechanism based on dense cue guidance. Use voxel block-level dense cue embedding vectors to generate cue fusion gating coefficients through gating mapping. Use the cue fusion gating coefficients to dynamically weight and fuse cue fusion features and self-attention features to generate the output feature sequence of the corresponding layer.

[0043] S46. After rearrangement and upsampling, the output features generate a global context feature map corresponding to each voxel of the standardized lung CT image sequence.

[0044] Optionally, S5 includes:

[0045] S51. Input the global context feature map into the multi-head semantic decoding module to perform shared decoding mapping and obtain the shared decoding feature map;

[0046] S52. In the multi-head semantic decoding module, construct nodule semantic head, cavity semantic head, fiber semantic head and background semantic head respectively. Input the shared decoding feature vector of the shared decoding feature map at the voxel coordinates into the nodule semantic head, cavity semantic head, fiber semantic head and background semantic head respectively to perform category scoring mapping, and obtain nodule category score, cavity category score, fiber category score and background category score respectively;

[0047] S53. By selecting the category type corresponding to the maximum value among the nodule category score, cavity category score, fiber category score and background category score at the voxel coordinates, the corresponding category type is used as the initial segmentation mask at the voxel coordinates.

[0048] Optionally, S6 includes:

[0049] S61. Obtain the initial segmentation mask and probability maps of each category, and identify the initial boundary voxel set in the initial segmentation mask;

[0050] S62. In the boundary refinement module, calculate the HU gradient magnitude for each voxel coordinate for the standardized HU value;

[0051] S63. For the nodule probability map, cavity probability map, and fiber probability map, calculate the nodule probability gradient magnitude, cavity probability gradient magnitude, and fiber probability gradient magnitude for each voxel coordinate, respectively.

[0052] S64. Divide the HU gradient vector of the voxel by the HU gradient magnitude to obtain the standardized HU gradient direction unit vector, and calculate the gradient consistency score based on the nodule probability gradient magnitude, cavity probability gradient magnitude and fiber probability gradient magnitude.

[0053] S65. Traverse all voxel coordinates in the 3D neighborhood. For a neighboring voxel belonging to the initial boundary voxel set, calculate the absolute value of the dot product of the current voxel and the unit vector of the standardized HU gradient direction of the neighboring voxel. Average all the absolute values ​​of the dot products that meet the conditions to obtain the edge continuity score of the voxel coordinates.

[0054] S66. The gradient consistency score and the edge continuity score are weighted and summed to obtain the boundary enhancement response value in voxel coordinates;

[0055] S67. Compare the boundary enhancement response value of each initial boundary voxel with the set boundary enhancement threshold to generate a boundary enhancement mask.

[0056] Optionally, S7 includes:

[0057] S71. In the linkage morphology reconstruction module, based on the category label of the boundary enhancement mask, extract the set of all voxel coordinates labeled as nodule category, the set of all voxel coordinates labeled as cavity category, and the set of all voxel coordinates labeled as fiber category respectively.

[0058] S72. Based on the extracted voxel coordinate sets of nodule category, cavity category, and fiber category, construct a lesion morphology atlas;

[0059] S73. For each voxel cluster in the lesion morphology atlas, based on the 26-neighborhood connection rule in three-dimensional space, spatially connected voxels of the same category are automatically clustered into independent voxel clusters.

[0060] S74. Calculate the morphological consistency score for each voxel cluster and perform morphological consistency screening;

[0061] S75. After completing the morphological consistency screening, for each voxel in all remaining voxel clusters, re-label it according to the voxel cluster category as the corresponding lesion category, and generate a morphological atlas correction mask.

[0062] Optionally, the morphological consistency screening includes:

[0063] If a voxel cluster is a nodule category and its morphological consistency score is lower than the nodule category morphological consistency threshold, then the category label of all voxels in the corresponding voxel cluster is set to the background category.

[0064] If a voxel cluster is a hole category and its morphological consistency score is lower than the morphological consistency threshold for the hole category, then the category label of all voxels in the corresponding voxel cluster is set to the background category.

[0065] If a voxel cluster belongs to the fiber category and its morphological consistency score is lower than the fiber category morphological consistency threshold, then the category label of all voxels in the corresponding voxel cluster is set to the background category.

[0066] The beneficial effects of this invention are:

[0067] This method employs a structured multi-category prompting mechanism and a global contextual feature fusion strategy. In the segmentation process, it utilizes regional structural feature extraction based on automatic prompting points and dynamic category prior generation to automatically identify various complex lesions such as nodules, cavities, and fibers. It also adaptively adjusts the segmentation response by combining category-related kernel functions. In the linked morphological reconstruction stage, it automatically identifies spatially adherent regions by constructing lesion morphological maps and category topology templates. Furthermore, it performs semantic stripping on adherent voxels based on three-dimensional connectivity and morphological consistency, accurately distinguishing spatially adjacent but semantically different lesions that were originally difficult to segment. This significantly reduces the missegmentation rate between nodules and fibers, and cavities and fibrous structures, overcoming the shortcomings of existing methods such as blurred boundaries and confusion of pathological categories.

[0068] This invention introduces a morphological connectivity algorithm based on topological template constraints to structurally complete spatial breaks between voxel clusters of the same category caused by image tomography, noise, or lesion development. By analyzing the spatial orientation and closure of fiber, cavity, and nodule voxel clusters, it automatically detects broken regions with a tendency for structural continuity. Under the condition of consistent connectivity and orientation, it performs three-dimensional voxel interpolation and relabeling, effectively repairing segmentation breaks such as fiber breaks and cavity ring gaps. This ensures that the final lesion structure assembly maintains anatomical consistency and integrity in three-dimensional space, which is significantly better than traditional morphological processing. It avoids structural contamination caused by indiscriminate expansion and improves the topological rationality of the segmentation results. Attached Figure Description

[0069] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0070] Figure 1 This is a flowchart of a lung image segmentation method based on an improved SAM2 proposed in this invention. Detailed Implementation

[0071] Example 1: Reference Figure 1 A lung image segmentation method based on an improved SAM2 includes:

[0072] S1. Acquire lung CT image sequences and preprocess them to obtain standardized lung CT image sequences;

[0073] In this embodiment, the preprocessing includes noise suppression, lung field region extraction, and grayscale value standardization.

[0074] S2. Input the standardized lung CT image sequence into the saliency detection module, generate a saliency heatmap based on Hounsfield Unit distribution, texture gradient information and spatial consistency rules, extract the center of candidate lesion region on the saliency heatmap and form an automatic prompting point set;

[0075] In this embodiment, S2 includes:

[0076] S21. Based on the normalized membership function of the standardized HU value within the preset density range, extract the high-density response terms corresponding to nodules and fibrous tissues and the low-density response terms corresponding to cavities from the standardized lung CT image sequence, and then weight and fuse them to obtain the density distribution response map.

[0077] In Example 1, the standardized lung CT image sequence is composed of multiple standardized lung CT image slices in sequence. Each standardized lung CT image slice has voxel coordinates in a three-dimensional coordinate system, and each voxel coordinate has a standardized HU value. The dimension of the standardized HU value is HU.

[0078] Normalized membership degree calculations were performed on the standardized HU value of each voxel in the standardized lung CT image sequence. When the standardized HU value was in the high-density range for nodules and fibrous tissues, the normalized membership degree of the corresponding voxel was used as the value of the high-density response term. When the standardized HU value was in the low-density range for cavities, the normalized membership degree of the corresponding voxel was used as the value of the low-density response term. Normalized membership degree calculations were performed on all voxels in the entire standardized lung CT image sequence to obtain the high-density response term map and the low-density response term map.

[0079] S22. Based on the gradient components of the standardized HU value in the x, y, and z directions of the three-dimensional coordinate system, calculate the three-dimensional gradient magnitude at each voxel coordinate, and use the three-dimensional gradient magnitude as the texture gradient response of the voxel to obtain the texture gradient response map.

[0080] S23. Using the coordinates of each voxel in the three-dimensional neighborhood of the voxel as a reference, sum the numerical differences between the density distribution response map and the texture gradient response map for each voxel to obtain the spatial consistency penalty term. Apply a negative exponential mapping to the spatial consistency penalty term based on the spatial consistency suppression coefficient to obtain the spatial consistency weight map.

[0081] S24. After normalizing the density distribution response map and texture gradient response map respectively, perform weighted summation and multiply it with the spatial consistency weight map to generate a significant heatmap;

[0082] S25. Apply a significance threshold to the significance heatmap, and aggregate voxels above the significance threshold into connected regions in three-dimensional space. The connected regions are used as candidate lesion regions.

[0083] All candidate lesion regions form a candidate lesion region set, and the number of candidate lesion regions is automatically determined by the voxel distribution.

[0084] S26. The weighted average of the three-dimensional coordinates is calculated by weighting the significance intensity of each voxel within the candidate lesion region, and the center of the candidate lesion region is taken as the center of the candidate lesion region. The centers of all candidate lesion regions form an automatic prompting point set.

[0085] In Example 1, all voxel coordinates within the candidate lesion region are traversed. The saliency intensity corresponding to each voxel coordinate is used as a weight. The coordinate value of the voxel coordinate in the three-dimensional coordinate system is multiplied with the corresponding saliency intensity. The product results of all voxel coordinates within the candidate lesion region are summed in three coordinate directions respectively. The summation result is divided by the sum of the saliency intensities of all voxels within the candidate lesion region to obtain a unique three-dimensional coordinate value. The three-dimensional coordinate value is determined as the center of the candidate lesion region corresponding to the candidate lesion region. The centers of all candidate lesion regions together constitute the set of automatic prompting points.

[0086] S3. Input the standardized lung CT image sequence and the set of auto-cue points into the medical cue encoder of the improved SAM2 model to generate multi-category cue maps;

[0087] In this embodiment, S3 includes:

[0088] S31. Input the standardized lung CT image sequence and the set of auto-cue points into the medical cue encoder of the improved SAM2 model. For each auto-cue point coordinate, the medical cue encoder selects voxels within a specified range to form a local voxel neighborhood centered on the auto-cue point coordinates.

[0089] The extent of a local voxel neighborhood is defined by Chebyshev distance and neighborhood radius.

[0090] S32. The medical prompt encoder obtains the mean HU and standard deviation HU based on the standardized HU values ​​of all voxels in the local voxel neighborhood;

[0091] S33. Based on the standardized HU value of each voxel in the local voxel neighborhood, calculate its gradient components in the x, y, and z directions respectively, and average the L2 norm of the gradient components of all voxels in the local voxel neighborhood to obtain the local gradient magnitude statistics.

[0092] S34. Calculate the structure tensor using the gradient vectors of voxels in the local voxel neighborhood;

[0093] In Example 1, for each local voxel neighborhood, the medical prompt encoder calculates the gradient components of each voxel in the x, y, and z directions based on the standardized HU value, forming a voxel gradient vector in a three-dimensional coordinate system. For the local voxel neighborhood, the encoder performs an outer product calculation on the gradient vectors of all voxels in the form of three-dimensional column vectors, obtaining the gradient vector outer product matrix corresponding to each voxel. The medical prompt encoder sums the gradient vector outer product matrices of all voxels in the neighborhood and divides the sum by the number of voxels contained in the local voxel neighborhood to obtain the structure tensor of the local voxel neighborhood. The structure tensor is a third-order symmetric real matrix.

[0094] S35. Using the structural tensor, calculate the morphological anisotropy index and the tubular uniformity index;

[0095] In Example 1, the medical prompting encoder performs eigenvalue decomposition on the structural tensor of each local voxel neighborhood after obtaining the corresponding structural tensor, resulting in three eigenvalues ​​sorted by numerical value. The first eigenvalue represents the intensity of structural change in the principal direction within the local voxel neighborhood, and the third eigenvalue represents the intensity of the weakest structural change orthogonal to the principal direction. Based on the ratio of the third eigenvalue to the first eigenvalue, the medical prompting encoder constructs a morphological anisotropy index. The morphological anisotropy index characterizes the degree of difference in structural change between the principal and non-principal directions within the local voxel neighborhood, reflecting whether the local voxel neighborhood exhibits a significant directional structural characteristic. The morphological anisotropy index corresponds to a large value when the third eigenvalue is small relative to the first eigenvalue, and a small value when the third eigenvalue is close to the first eigenvalue. The medical encoder utilizes the difference between the first and second eigenvalues ​​to construct a tubular consistency index. The tubular consistency index measures whether a local voxel neighborhood has tubular or linear continuous structural features by characterizing whether structural changes are concentrated along a single principal direction. When the first eigenvalue is significantly greater than the second eigenvalue, the tubular consistency index corresponds to a large value; when the first and second eigenvalues ​​are close, the tubular consistency index corresponds to a small value.

[0096] S36. The medical prompt encoder uses HU mean, HU standard deviation, local gradient magnitude statistics, morphological anisotropy index and tubular consistency index as input features to form a category prior vector;

[0097] In Example 1, the HU mean, HU standard deviation, local gradient magnitude statistics, morphological anisotropy index, and tubular consistency index are combined in a preset order to form a local feature vector. The local feature vector is used to uniformly represent the density level, density stability, structural change intensity, and spatial morphological characteristics of the local voxel neighborhood where the automatic prompting point is located. The medical prompting encoder inputs the local feature vector into the category mapping unit. The category mapping unit performs a linear weighted transformation on the local feature vector and applies a bias term to the weighted transformation result to obtain three types of response values ​​for nodule category, cavity category, and fibrous category. Each type of response value is used to measure the relative matching degree of the automatic prompting point under the corresponding lesion category. The medical prompting encoder performs normalization mapping processing on the three types of response values, so that the three types of response values ​​are constrained to be non-negative and their sum is 1. The three types of response values ​​after normalization mapping processing are used as the prior weights of nodule category, cavity category, and fibrous category, respectively. The three together constitute the category prior vector of the automatic prompting point.

[0098] S37. Based on the local gradient magnitude statistics and HU standard deviation, calculate the confidence weight of each automatic prompt point using a weighted normalization function;

[0099] In Example 1, the medical prompt encoder applies a positive weighting coefficient to the normalized local gradient magnitude statistic and a suppression weighting coefficient to the normalized HU standard deviation, and performs a weighted combination to obtain the prompt reliability response value. A monotonically normalized mapping is applied to the prompt reliability response value so that the prompt reliability response value is mapped to a preset interval, and the mapped value is determined as the prompt confidence weight of the automatic prompt point. The prompt confidence weight is used to represent the structural stability and prompt credibility of the automatic prompt point in the complex lung multiple lesion segmentation task.

[0100] S38. For each auto-suggestion point, generate a morphologically adaptive class-related kernel function according to the class type based on the feature vector information of the class prior vector, suggestion confidence weight, and structure tensor.

[0101] In Example 1, the target lesion category type corresponding to the automatic prompt point is determined based on the category prior weight with the largest proportion in the category prior vector. Then, the corresponding kernel function construction mode is selected according to the determined target lesion category type. When the target lesion category type is nodule, the isotropic kernel function construction mode is selected; when the target lesion category type is cavity, the annular kernel function construction mode is selected; and when the target lesion category type is fibrous, the anisotropic kernel function construction mode is selected.

[0102] When constructing isotropic kernel functions, the kernel functions with consistent weights in all directions are generated in three-dimensional space, centered on the coordinates of the automatically prompted point and according to preset spatial scale parameters. When constructing cyclic kernel functions, inner and outer kernel functions are generated separately, centered on the coordinates of the automatically prompted point, and a cyclic kernel function with a hollow structure is formed by differentiating the outer and inner kernel functions. When constructing anisotropic kernel functions, the kernel function is stretched along the principal direction and compressed in a direction orthogonal to the principal direction, using the first eigenvector of the structure tensor as the principal direction, forming a spatial kernel function that extends along the principal direction.

[0103] In the process of constructing three kernel functions, the medical prompt encoder modulates the overall amplitude of the kernel function according to the prompt confidence weight, so that the kernel function corresponding to the automatic prompt point with a higher prompt confidence weight has a stronger response intensity in space, and the kernel function corresponding to the automatic prompt point with a lower prompt confidence weight has a weaker response intensity in space, thus obtaining a morphologically adaptive category-related kernel function for the automatic prompt point.

[0104] S39. The medical prompt encoder uses the prior weights of all automatic prompt points, the prompt confidence weights, and the class correlation kernel function to weight and superimpose the three types of responses of all voxels in a standardized lung CT image sequence to obtain a multi-class prompt heat volume.

[0105] In Example 1, after constructing the morphologically adaptive category-related kernel function for each automatic prompting point, the medical prompt encoder calculates the prompting response intensity of each voxel coordinate in the standardized lung CT image sequence under the nodule, cavity, and fibrosis categories. For any voxel coordinate in the standardized lung CT image sequence, the medical prompt encoder, based on the spatial relationship between the voxel coordinate and the coordinates of each automatic prompting point, calls the category-related kernel function of the corresponding automatic prompting point to calculate the spatial response value of the voxel under the action of that automatic prompting point, thus providing a prompting response for each automatic prompting point. The spatial response value generated by the point is multiplied by the prior weight of the category and the confidence weight of the prompt, respectively, to jointly modulate the tendency of different lesion categories and the reliability of the prompt, thus obtaining the weighted prompt response value of the automatic prompt point for the voxel in the corresponding category. For the same voxel coordinates, the weighted prompt response values ​​generated by all automatic prompt points are accumulated under the nodule category, cavity category and fibrous category, respectively, to form the three cumulative prompt response values ​​of the voxel under the nodule category, cavity category and fibrous category. The three cumulative prompt response values ​​correspond one-to-one with the standardized lung CT image sequence in space.

[0106] The medical prompt encoder organizes the cumulative prompt response values ​​of all voxels under the nodule, cavity, and fiber categories into a three-channel multi-category prompt heat volume. Each channel corresponds to the prompt heat volume for the nodule category, the prompt heat volume for the cavity category, and the prompt heat volume for the fiber category, respectively.

[0107] S310. Generate a dense cue embedding tensor by passing the cue embedding mapping of the multi-category cue heat volume through the cue embedding map of the medical cue encoder;

[0108] The dense cue embedding tensor maintains a one-to-one correspondence with the standardized lung CT image sequence in space, and has a set number of cue embedding dimensions;

[0109] In Example 1, the medical prompt encoder reads the voxel distribution of the multi-category prompt heat volume in three-dimensional space. Using voxel coordinates as an index, it performs joint encoding processing on the multi-category prompt heat value at each voxel position in the multi-category prompt heat volume. The joint encoding processing includes concatenating the prompt heat values ​​of nodule category, cavity category, and fiber category according to a fixed category order to form a voxel-level prompt feature vector. The voxel-level prompt feature vector is input into the prompt embedding mapping unit, and a voxel-by-voxel mapping operation is performed on the voxel-level prompt feature vector. The voxel-by-voxel mapping operation projects the voxel-level prompt feature vector into dimensions through a learnable linear transformation, and applies nonlinear activation processing to the projection result to map the voxel-level prompt feature vector into a prompt embedding vector of fixed dimension. The voxel-by-voxel mapping operation is repeated for all voxel positions in the standardized lung CT image sequence, and all prompt embedding vectors are arranged according to the spatial order of voxels in the three-dimensional coordinate system to generate a dense prompt embedding tensor that is spatially aligned voxel-by-voxel with the standardized lung CT image sequence.

[0110] S311. The medical prompt encoder generates point location encoding vector, category prior encoding vector, and confidence encoding vector respectively based on the set of automatic prompt points, category prior vector, and prompt confidence weight, and then fuses them to output a sparse prompt embedding vector for each automatic prompt point;

[0111] In Example 1, the medical prompt encoder fuses the point location encoding vector, the category prior encoding vector, and the confidence encoding vector in a fixed order, including aligning the three types of encoding vectors dimension by dimension and then adding them together to obtain the sparse prompt embedding vector corresponding to the automatic prompt point.

[0112] S312. The medical cue encoder unifies the arrangement of dense cue embedding tensors and sparse cue embedding vectors to form a multi-category cue graph.

[0113] In Example 1, the medical prompt encoder uses the three-dimensional voxel coordinate system of the standardized lung CT image sequence as a unified spatial reference coordinate system. Each prompt embedding vector in the dense prompt embedding tensor is spatially positioned according to its corresponding voxel coordinates, aligning the dense prompt embedding tensor with the standardized lung CT image sequence voxel-by-voxel in three-dimensional space. For each sparse prompt embedding vector in the sparse prompt embedding set, based on the spatial coordinates of its corresponding auto-prompt point in the three-dimensional coordinate system, the sparse prompt embedding vector is bound to its corresponding spatial position, forming a sparse prompt embedding representation in the same spatial reference coordinate system as the dense prompt embedding tensor. After spatial alignment, the dense prompt embedding tensor and the sparse prompt embedding vector at the corresponding spatial position are concatenated along the feature dimension to form a joint prompt feature vector containing both dense and sparse prompt information. All voxel positions are rearranged according to the voxel arrangement order in three-dimensional space, generating a multi-category prompt image that is consistent with the standardized lung CT image sequence in spatial structure and simultaneously contains both dense and sparse prompt information in the feature dimension.

[0114] S4. Input the standardized lung CT image sequence into the image encoder of the improved SAM2 model, and fuse the multi-class cue map into the encoder feature path to generate a global context feature map;

[0115] In this embodiment, S4 includes:

[0116] S41. Input the standardized lung CT image sequence into the image encoder of the improved SAM2 model to perform voxel block segmentation and feature aggregation to form an image voxel block embedding sequence;

[0117] The standardized lung CT image sequence is divided into voxel blocks in sequence. Each voxel block consists of a set of continuous voxels. By aggregating the features of all voxels in each voxel block, voxel block embedding vectors are obtained. All voxel block embedding vectors are arranged in the order of the three-dimensional voxel blocks to form an image voxel block embedding sequence.

[0118] S42. Based on the voxel block partitioning method, aggregate the dense cue embedding vectors of all voxel positions in each voxel block of the dense cue embedding tensor to form a voxel block-level dense cue embedding vector.

[0119] Each voxel block is aggregated to form a voxel block-level dense cue embedding vector. All voxel block-level dense cue embedding vectors are arranged in the order of voxel blocks to form a voxel block-level dense cue embedding sequence, ensuring that each voxel block corresponds to a unique voxel block-level dense cue embedding vector.

[0120] S43. For each feature layer of the image encoder, perform self-attention encoding on the output feature sequence of the previous layer to extract the self-attention features of the image itself;

[0121] The image voxel block embedding sequence is used as the initial input feature sequence for each feature layer of the image encoder. Each feature vector in the initial input feature sequence corresponds one-to-one with the corresponding voxel block, and the number of dimensions of all feature vectors is consistent with the number of dimensions of the cue embedding.

[0122] S44. Construct a dual-source cue cross-attention mechanism, using the self-attention features of the image itself as the query, and the voxel block-level dense cue embedding sequence and sparse cue embedding vector set as the key and value, respectively calculate the attention response of image features to dense cues and sparse cues, and aggregate them to obtain cue fusion features;

[0123] All sparse cue embedding vectors in the multi-category cue graph are arranged into a sparse cue embedding vector set according to the spatial coordinates of the auto-cue points. The number of sparse cue embedding vectors is consistent with the number of auto-cue points, and the spatial index of each sparse cue embedding vector is completely consistent with the position of the corresponding auto-cue point in the three-dimensional voxel coordinate system.

[0124] S45. Construct an adaptive gating mechanism based on dense cue guidance. Use voxel block-level dense cue embedding vectors to generate cue fusion gating coefficients through gating mapping. Use the cue fusion gating coefficients to dynamically weight and fuse cue fusion features and self-attention features to generate the output feature sequence of the corresponding layer.

[0125] Gating coefficients are used to enhance the weight of fused features in regions where the cue information is clear, while maintaining the dominance of the original image features in regions where the cue information is weak.

[0126] S46. After rearrangement and upsampling, the output features generate a global context feature map corresponding to each voxel of the standardized lung CT image sequence.

[0127] In Example 1, after all feature layers are calculated, the output feature sequence of the last layer is rearranged according to the spatial arrangement of voxel blocks. The voxel spatial resolution is restored to be consistent with the standardized lung CT image sequence through upsampling. The feature vectors output by all voxel positions are arranged in spatial order to form a global context feature map that corresponds one-to-one with the space of the standardized lung CT image sequence.

[0128] S5. Input the global context feature map into the multi-head semantic decoding module, output the nodule probability map, hole probability map and fiber probability map according to the nodule semantic head, hole semantic head and fiber semantic head respectively, and generate a preliminary segmentation mask based on the nodule probability map, hole probability map and fiber probability map;

[0129] In this embodiment, S5 includes:

[0130] S51. Input the global context feature map into the multi-head semantic decoding module to perform shared decoding mapping and obtain the shared decoding feature map;

[0131] The shared decoded feature map corresponds to the global context feature map on a voxel-by-voxel basis in space. For any voxel coordinate, the shared decoded feature map outputs the shared decoded feature vector at the corresponding voxel coordinate. The dimension of the shared decoded feature vector is the same as that of the voxel-level context feature vector.

[0132] S52. In the multi-head semantic decoding module, construct nodule semantic head, cavity semantic head, fiber semantic head and background semantic head respectively. Input the shared decoding feature vector of the shared decoding feature map at the voxel coordinates into the nodule semantic head, cavity semantic head, fiber semantic head and background semantic head respectively to perform category scoring mapping, and obtain nodule category score, cavity category score, fiber category score and background category score respectively;

[0133] In Example 1, the nodule category score is obtained by linearly transforming the category score weight vector of the nodule semantic head and the shared decoded feature vector, and then adding the category score bias term of the nodule semantic head; the cavity category score is obtained by linearly transforming the category score weight vector of the cavity semantic head and the shared decoded feature vector, and then adding the category score bias term of the cavity semantic head; the fiber category score is obtained by linearly transforming the category score weight vector of the fiber semantic head and the shared decoded feature vector, and then adding the category score bias term of the fiber semantic head; the background category score is obtained by linearly transforming the category score weight vector of the background semantic head and the shared decoded feature vector, and then adding the category score bias term of the background semantic head. The category score weight vectors and category score bias terms of the nodule semantic head, cavity semantic head, fiber semantic head, and background semantic head are all learnable parameters.

[0134] S53. By selecting the category type corresponding to the maximum value among the nodule category score, cavity category score, fiber category score and background category score at the voxel coordinates, the corresponding category type is used as the initial segmentation mask at the voxel coordinates.

[0135] The initial segmentation mask is a voxel-level category label map, which spatially corresponds to a voxel-by-voxel of a standardized lung CT image sequence. When the maximum category score corresponds to the nodule category, the corresponding voxel is labeled as the nodule category; when the maximum category score corresponds to the cavity category, the corresponding voxel is labeled as the cavity category; when the maximum category score corresponds to the fiber category, the corresponding voxel is labeled as the fiber category; when the maximum category score corresponds to the background category, the corresponding voxel is labeled as the background category.

[0136] S6. Input the preliminary segmentation mask, nodule probability map, hole probability map and fiber probability map into the boundary refinement module, and generate a boundary enhancement mask based on gradient consistency rules and edge continuity rules;

[0137] In this embodiment, S6 includes:

[0138] S61. Obtain the initial segmentation mask and probability maps of each category, and identify the initial boundary voxel set in the initial segmentation mask;

[0139] In the boundary refinement module, based on the preliminary segmentation mask, for each voxel coordinate, it is determined whether there are adjacent voxels with different category labels in the three-dimensional neighborhood. If so, the voxel coordinate is included in the initial boundary voxel set. The voxels in the initial boundary voxel set correspond one-to-one with the standardized lung CT image sequence in spatial position.

[0140] S62. In the boundary refinement module, calculate the HU gradient magnitude for each voxel coordinate for the standardized HU value;

[0141] In the boundary refinement module, for the standardized HU value, the gradient components of each voxel coordinate in the x, y, and z directions are calculated, and the square root of the sum of the squares of the gradient components in the three directions is taken to obtain the HU gradient magnitude of each voxel coordinate.

[0142] S63. For the nodule probability map, cavity probability map, and fiber probability map, calculate the nodule probability gradient magnitude, cavity probability gradient magnitude, and fiber probability gradient magnitude for each voxel coordinate, respectively.

[0143] For the nodule probability map, cavity probability map, and fiber probability map, calculate the gradient components of each voxel coordinate in the x, y, and z directions respectively. Then, take the square root of the sum of the squares of the gradient components in the three directions of each category probability map to obtain the nodule probability gradient magnitude, cavity probability gradient magnitude, and fiber probability gradient magnitude of each voxel coordinate.

[0144] S64. Divide the HU gradient vector of the voxel by the HU gradient magnitude to obtain the standardized HU gradient direction unit vector, and calculate the gradient consistency score based on the nodule probability gradient magnitude, cavity probability gradient magnitude and fiber probability gradient magnitude.

[0145] In Example 1, for the nodule probability map, cavity probability map, and fiber probability map, the probability gradient vector of each type of probability map is calculated at voxel coordinates. The probability gradient vector is then divided by the sum of the corresponding probability gradient magnitude and the stabilization term to obtain the unit vectors of the nodule probability gradient direction, cavity probability gradient direction, and fiber probability gradient direction. For the nodule, cavity, and fiber categories, the directional consistency response between the normalized HU gradient direction unit vector and the corresponding category probability gradient direction unit vector is calculated. This directional consistency response is then jointly modulated with the normalized probability gradient magnitude and the normalized HU gradient magnitude of the corresponding category to obtain the three gradient consistency candidate scores for the voxel under the nodule, cavity, and fiber categories. The maximum value among the three gradient consistency candidate scores for the nodule, cavity, and fiber categories is selected as the gradient consistency score at the voxel coordinates, representing the boundary response intensity that best matches the actual image boundary change direction under multi-lesion semantic conditions.

[0146] S65. Traverse all voxel coordinates in the 3D neighborhood. For a neighboring voxel belonging to the initial boundary voxel set, calculate the absolute value of the dot product of the current voxel and the unit vector of the standardized HU gradient direction of the neighboring voxel. Average all the absolute values ​​of the dot products that meet the conditions to obtain the edge continuity score of the voxel coordinates.

[0147] S66. The gradient consistency score and the edge continuity score are weighted and summed to obtain the boundary enhancement response value in voxel coordinates;

[0148] S67. Compare the boundary enhancement response value of each initial boundary voxel with the set boundary enhancement threshold to generate a boundary enhancement mask.

[0149] In Example 1, if the boundary enhancement response value is greater than or equal to the boundary enhancement threshold, the category label of the corresponding voxel coordinate in the boundary enhancement mask is set as the category label of the initial segmentation mask at the voxel coordinate; if the boundary enhancement response value is less than the boundary enhancement threshold, or the voxel does not belong to the initial boundary voxel set, its category label is set as the background category.

[0150] S7. Input the boundary enhancement mask into the linkage morphology reconstruction module, construct the lesion morphology map based on the nodule topology template, cavity topology template and fiber topology template, and perform three-dimensional connectivity analysis on the lesion morphology map to obtain the morphology map correction mask;

[0151] In this embodiment, S7 includes:

[0152] S71. In the linkage morphology reconstruction module, based on the category label of the boundary enhancement mask, extract the set of all voxel coordinates labeled as nodule category, the set of all voxel coordinates labeled as cavity category, and the set of all voxel coordinates labeled as fiber category respectively.

[0153] Each set of voxel coordinates contains only voxels labeled with the same category, and each voxel coordinate can belong to only one category set.

[0154] S72. Based on the extracted voxel coordinate sets of nodule category, cavity category, and fiber category, construct a lesion morphology atlas;

[0155] In Example 1, the lesion morphology atlas consists of a node set and an edge set. The node set represents all voxel clusters in three-dimensional space, and each voxel cluster is composed of spatially connected voxels of the same type. The edge set represents the connection relationship established between different nodes (different voxel clusters) based on the three-dimensional spatial adjacency relationship and anatomical morphological features.

[0156] In constructing lesion morphology atlases, for nodule-type voxel clusters, a closed spherical structural template is used for structural coding. The structural coding process uses the spatial connectivity and local volume compactness of each nodule-type voxel cluster as features. For cavity-type voxel clusters, a hollow ring-shaped structural template is used for structural coding. The structural coding process uses the closed boundary characteristics and low-density cavity characteristics of the central region of each cavity-type voxel cluster as features. For fibrous-type voxel clusters, a slender branch-shaped structural template is used for structural coding. The structural coding process uses the high orientation and length continuity of each fibrous-type voxel cluster as features. Each structural code is based on the consistency between the spatial morphology of the voxel cluster and the category structural template.

[0157] S73. For each voxel cluster in the lesion morphology atlas, based on the 26-neighborhood connection rule in three-dimensional space, spatially connected voxels of the same category are automatically clustered into independent voxel clusters.

[0158] All voxel clusters cover all non-background voxels in the lesion morphology atlas, and each voxel belongs to only one voxel cluster.

[0159] S74. Calculate the morphological consistency score for each voxel cluster and perform morphological consistency screening;

[0160] In Example 1, the morphological consistency score is used to measure the degree of morphological matching between a voxel cluster and the topological template of its category. The morphological consistency score is obtained by weighted summation of the geometric compactness index of the voxel cluster and the morphological similarity between the voxel cluster and the category topological template. The geometric compactness index is used to measure the compactness of the voxel cluster in three-dimensional space, and the morphological similarity is used to measure the similarity between the voxel cluster and the corresponding category topological template in spatial structure.

[0161] The geometric compactness index is obtained by analyzing the voxel distribution relationship of voxel clusters in three-dimensional space. When the voxel clusters are more concentrated in space and have fewer internal gaps, the geometric compactness index value is larger. When the voxel clusters are more dispersed in space and have more internal gaps, the geometric compactness index value is smaller.

[0162] The morphological similarity between voxel clusters and category topological templates is obtained by comparing the spatial structural features of voxel clusters with the expected structural features of the corresponding category topological templates.

[0163] In this embodiment, the morphological consistency screening includes:

[0164] If a voxel cluster is a nodule category and its morphological consistency score is lower than the nodule category morphological consistency threshold, then the category label of all voxels in the corresponding voxel cluster is set to the background category.

[0165] If a voxel cluster is a hole category and its morphological consistency score is lower than the morphological consistency threshold for the hole category, then the category label of all voxels in the corresponding voxel cluster is set to the background category.

[0166] If a voxel cluster belongs to the fiber category and its morphological consistency score is lower than the fiber category morphological consistency threshold, then the category label of all voxels in the corresponding voxel cluster is set to the background category.

[0167] S75. After completing the morphological consistency screening, for each voxel in all remaining voxel clusters, re-label it according to the voxel cluster category as the corresponding lesion category, and generate a morphological atlas correction mask.

[0168] The morphological atlas correction mask is a class label map aligned to the voxel space of a normalized lung CT image sequence.

[0169] S8. Perform semantic stripping on the adhered regions of the morphological map correction mask and perform morphological connection on the broken regions to obtain the topological consistency optimized mask. Map the topological consistency optimized mask to the three-dimensional coordinate system and perform voxel-by-voxel fusion with the standardized lung CT image sequence to reconstruct the three-dimensional lesion structure set.

[0170] In Example 1, based on the morphological map correction mask, the spatial adjacency relationship between different types of voxel clusters is analyzed. When different types of voxel clusters share neighboring voxels or adjacent voxels in the three-dimensional neighborhood, the voxels are identified as adhesion voxels. For adhesion voxels, semantic stripping operation is performed on the adhesion voxels according to their three-dimensional connectivity, connectivity direction consistency and matching degree with the category topology template in different types of voxel clusters. The adhesion voxels are re-marked as lesion categories with higher connectivity and topological consistency.

[0171] Simultaneously, for voxel cluster pairs that are spatially spaced but have a continuous trend under directional and topological template constraints, morphological connection operations are performed. Under the premise of satisfying distance threshold and directional consistency constraints, connected voxels are inserted between voxel clusters to restore structural continuity. After semantic stripping and morphological connection processing, a topological consistency optimization mask is generated and mapped to a three-dimensional coordinate system. This mask is then fused with a standardized lung CT image sequence voxel by voxel to reconstruct a set of three-dimensional lesion structures.

[0172] Example 2: In an automated analysis process, the operator retrieved a CT sequence (number "P1543") from the lung imaging database, which is a typical case of complex multiple lesions. The volume of the patient's three-dimensional lung CT data is 512×512×240 voxels. Manual labeling results showed that there were 5 nodules, 3 cavities, and 4 fibrotic cord areas in the image, and some nodules were adhered to fibers, and the cavity walls were wrapped and adhered to fibers.

[0173] The system performs noise suppression processing on the CT sequence. In the original CT image, the voxel mean square noise intensity was 42.1 HU, which was reduced to 11.3 HU after filtering. The lung field extraction module segmented a binary lung field mask, increasing the voxel coverage to 91.2%. Subsequently, the HU values ​​of all voxels were linearly standardized, with the distribution range standardized from the original [-1150, 450] to [-1, 1]. The standardized parameters of each layer were recorded to facilitate subsequent three-dimensional spatial consistency correction.

[0174] The system detected a local mean density of 0.58 and a density dispersion of 0.09 around voxel point (236, 315, 122), with this region scoring 0.92 in the HU density high-threshold normalized response map. Meanwhile, voxel point (402, 218, 87) had a local mean density of -0.61 and a normalized low-density response of 0.87, automatically identifying it as a suspected void region. After 3D spatial max pooling and saliency heatmap thresholding, the system automatically generated 12 alert point coordinates: 5 nodule alert points, 3 void alert points, and 4 fiber alert points. All alert points were centered at the centroid of the automatically clustered region.

[0175] Taking the cue point (236, 315, 122) as an example, within a local neighborhood of radius 6, the mean local density is 0.58, the density dispersion is 0.09, the average gradient magnitude is 0.17, the morphological anisotropy index is 0.81, and the tubular consistency index is 0.22. Based on linear weighting and softmax normalization, the model defines the prior vector for this point as [nodules 0.74, cavities 0.09, fibers 0.17], with a cue confidence weight of 0.93. The system generates an adaptive category-related kernel function for each cue point and superimposes it onto the corresponding 3D space. Ultimately, the peak value of the multi-category cue heatmap in space highly matches the manually labeled data.

[0176] Standardized CT data was divided into 8×8×4 voxel blocks, totaling 4096 voxel blocks. Each voxel was aggregated to generate a 128-dimensional feature vector. Dense cue embedding vectors were averaged over voxel blocks and then concatenated with the corresponding voxel blocks as the initial input to the encoder. After fusion of four layers of self-attention and cue cross-attention, the gating weight distribution of each layer was adaptively adjusted. In areas with clear cue information (such as nodules and cavitary cores), the average gating coefficient was 0.81, while in normal lung tissue areas, the average gating coefficient decreased to 0.19, effectively enhancing the model's response to key areas.

[0177] The global context feature map is fed into the multi-head decoding module. The voxel (236, 315, 122) scores 2.17 using the nodule semantic head, -1.09 for holes, 0.21 for fibers, and -1.65 for background. After softmax normalization, the nodule probability is 0.80, and the system initially labels it as a nodule. The voxel (402, 218, 87) has a hole probability of 0.89, and the system labels it as a hole. The average Dice of the nodule probability map and the manual label is 0.91, the hole probability map Dice is 0.88, and the fiber probability map Dice is 0.82. The initial segmentation mask generally matches the manual label highly, but there is still local confusion at the nodule-fiber interface.

[0178] The system detected 7856 boundary voxels in the initial mask (with class variations in their neighborhoods). The average HU gradient of these voxels was 0.19 HU / voxel. For voxel (254, 317, 120), the absolute value of the dot product of its HU gradient direction and nodule probability gradient direction was 0.93, the normalized gradient consistency score was 0.87, the neighborhood directional consistency score was 0.91, and the final boundary enhancement response value was 0.89, which is higher than the set threshold of 0.75, and was retained as a nodule boundary. Some fiber-nodule adhesion regions had an original response of 0.62, which was refined and classified as background, effectively improving the accuracy of the boundary.

[0179] The system extracted a total of 13,582 voxels for nodule categories, automatically identifying them as 5 spatially connected voxel clusters. Four of these clusters completely overlapped with manually labeled clusters, while the fifth partially overlapped with the fibrous region. 14,239 voxels were identified as 3 spatially closed loops for cavities, and 11,760 voxels were identified as 4 linear voxel clusters, one of which was interrupted. A morphological consistency score was automatically calculated for each voxel cluster: a minimum score of 0.84 for nodules (threshold 0.80), 0.81 for cavities (threshold 0.78), and 0.77 for fibers (threshold 0.75). All voxel clusters were retained, while a few geometrically discrete fragments (minimum score 0.48) were removed as background.

[0180] In the nodule-fiber adhesion region, the system detected 116 voxels belonging to the spatially adjacent region. After automatically traversing its three-dimensional neighborhood, it was found that 92 voxels had more connections to nodules than to fibers. The topology template consistency judgment was biased towards nodules, and the system automatically re-labeled them as nodules. For fiber fractures, the system searched for fracture endpoints with a spatial distance of less than 4 voxels and an angle of less than 20° in the principal direction. By interpolating, 37 voxels were completed, achieving closed-loop reconstruction of the fiber cluster. Finally, the consistency rate between the optimized topology mask and the manually corrected annotations was improved to 97.8%.

[0181] The voxel-level Dice coefficients for the full-process segmentation results were: nodules 0.93, cavities 0.89, and fibers 0.84; the segmentation breakage rate decreased to 2.5%, and the structural confusion rate was 3.2%. In comparison with the traditional nnU-Net for the same case, the nodule Dice was 0.87, the cavity 0.80, and the fiber 0.70, with a segmentation breakage rate of 8.1% and a structural confusion rate of 12.4%. The automatic segmentation and structural correction of this invention took 8 minutes, manual full-process correction took only 3 minutes, and the traditional method took 23 minutes.

[0182] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A lung image segmentation method based on an improved SAM2, characterized in that, include: S1. Acquire lung CT image sequences and preprocess them to obtain standardized lung CT image sequences; S2. Input the standardized lung CT image sequence into the saliency detection module, generate a saliency heatmap based on Hounsfield Unit distribution, texture gradient information and spatial consistency rules, extract the center of candidate lesion region on the saliency heatmap and form an automatic prompting point set; S3. Input the standardized lung CT image sequence and the set of auto-cue points into the medical cue encoder of the improved SAM2 model to generate multi-category cue maps; S4. Input the standardized lung CT image sequence into the image encoder of the improved SAM2 model, and fuse the multi-class cue map into the encoder feature path to generate a global context feature map; S5. Input the global context feature map into the multi-head semantic decoding module, output the nodule probability map, hole probability map and fiber probability map according to the nodule semantic head, hole semantic head and fiber semantic head respectively, and generate a preliminary segmentation mask based on the nodule probability map, hole probability map and fiber probability map; S6. Input the preliminary segmentation mask, nodule probability map, hole probability map and fiber probability map into the boundary refinement module, and generate a boundary enhancement mask based on gradient consistency rules and edge continuity rules; S7. Input the boundary enhancement mask into the linkage morphology reconstruction module, construct the lesion morphology map based on the nodule topology template, cavity topology template and fiber topology template, and perform three-dimensional connectivity analysis on the lesion morphology map to obtain the morphology map correction mask; S8. Perform semantic stripping on the adhered regions of the morphological map correction mask and perform morphological connection on the broken regions to obtain the topological consistency optimized mask. Map the topological consistency optimized mask to the three-dimensional coordinate system and perform voxel-by-voxel fusion with the standardized lung CT image sequence to reconstruct the three-dimensional lesion structure set.

2. The lung image segmentation method based on improved SAM2 according to claim 1, characterized in that, The preprocessing includes noise suppression, lung field region extraction, and grayscale value standardization.

3. The lung image segmentation method based on improved SAM2 according to claim 1, characterized in that, S2 includes: S21. Based on the normalized membership function of the standardized HU value within the preset density range, extract the high-density response terms corresponding to nodules and fibrous tissues and the low-density response terms corresponding to cavities from the standardized lung CT image sequence, and then weight and fuse them to obtain the density distribution response map. S22. Based on the gradient components of the standardized HU value in the x, y, and z directions of the three-dimensional coordinate system, calculate the three-dimensional gradient magnitude at each voxel coordinate, and use the three-dimensional gradient magnitude as the texture gradient response of the voxel to obtain the texture gradient response map. S23. Using the coordinates of each voxel in the three-dimensional neighborhood of the voxel as a reference, sum the numerical differences between the density distribution response map and the texture gradient response map for each voxel to obtain the spatial consistency penalty term. Apply a negative exponential mapping to the spatial consistency penalty term based on the spatial consistency suppression coefficient to obtain the spatial consistency weight map. S24. After normalizing the density distribution response map and texture gradient response map respectively, perform weighted summation and multiply it with the spatial consistency weight map to generate a significant heatmap; S25. Apply a significance threshold to the significance heatmap, and aggregate voxels above the significance threshold into connected regions in three-dimensional space. The connected regions are used as candidate lesion regions. S26. The weighted average of the three-dimensional coordinates is calculated by weighting the significance intensity of each voxel within the candidate lesion region, and the center of the candidate lesion region is taken as the center of the candidate lesion region. The centers of all candidate lesion regions form an automatic prompting point set.

4. The lung image segmentation method based on improved SAM2 according to claim 1, characterized in that, S3 includes: S31. Input the standardized lung CT image sequence and the set of auto-cue points into the medical cue encoder of the improved SAM2 model. For each auto-cue point coordinate, the medical cue encoder selects voxels within a specified range to form a local voxel neighborhood centered on the auto-cue point coordinates. S32. The medical prompt encoder obtains the mean HU and standard deviation HU based on the standardized HU values ​​of all voxels in the local voxel neighborhood; S33. Based on the standardized HU value of each voxel in the local voxel neighborhood, calculate its gradient components in the x, y, and z directions respectively, and average the L2 norm of the gradient components of all voxels in the local voxel neighborhood to obtain the local gradient magnitude statistics. S34. Calculate the structure tensor using the gradient vectors of voxels in the local voxel neighborhood; S35. Using the structural tensor, calculate the morphological anisotropy index and the tubular uniformity index; S36. The medical prompt encoder uses HU mean, HU standard deviation, local gradient magnitude statistics, morphological anisotropy index and tubular consistency index as input features to form a category prior vector; S37. Based on the local gradient magnitude statistics and HU standard deviation, calculate the confidence weight of each automatic prompt point using a weighted normalization function; S38. For each auto-suggestion point, generate a morphologically adaptive class-related kernel function according to the class type based on the feature vector information of the class prior vector, suggestion confidence weight, and structure tensor. S39. The medical prompt encoder uses the prior weights of all automatic prompt points, the prompt confidence weights, and the class correlation kernel function to weight and superimpose the three types of responses of all voxels in a standardized lung CT image sequence to obtain a multi-class prompt heat volume. S310. Generate a dense cue embedding tensor by passing the cue embedding mapping of the multi-category cue heat volume through the cue embedding map of the medical cue encoder; The dense cue embedding tensor maintains a one-to-one correspondence with the standardized lung CT image sequence in space, and has a set number of cue embedding dimensions; S311. The medical prompt encoder generates point location encoding vector, category prior encoding vector, and confidence encoding vector respectively based on the set of automatic prompt points, category prior vector, and prompt confidence weight, and then fuses them to output a sparse prompt embedding vector for each automatic prompt point; S312. The medical cue encoder unifies the arrangement of dense cue embedding tensors and sparse cue embedding vectors to form a multi-category cue graph.

5. The lung image segmentation method based on improved SAM2 according to claim 1, characterized in that, S4 includes: S41. Input the standardized lung CT image sequence into the image encoder of the improved SAM2 model to perform voxel block segmentation and feature aggregation to form an image voxel block embedding sequence; S42. Based on the voxel block partitioning method, aggregate the dense cue embedding vectors of all voxel positions in each voxel block of the dense cue embedding tensor to form a voxel block-level dense cue embedding vector. S43. For each feature layer of the image encoder, perform self-attention encoding on the output feature sequence of the previous layer to extract the self-attention features of the image itself; S44. Construct a dual-source cue cross-attention mechanism, using the self-attention features of the image itself as the query, and the voxel block-level dense cue embedding sequence and sparse cue embedding vector set as the key and value, respectively calculate the attention response of image features to dense cues and sparse cues, and aggregate them to obtain cue fusion features; S45. Construct an adaptive gating mechanism based on dense cue guidance. Use voxel block-level dense cue embedding vectors to generate cue fusion gating coefficients through gating mapping. Use the cue fusion gating coefficients to dynamically weight and fuse cue fusion features and self-attention features to generate the output feature sequence of the corresponding layer. S46. After rearrangement and upsampling, the output features generate a global context feature map corresponding to each voxel of the standardized lung CT image sequence.

6. The lung image segmentation method based on improved SAM2 according to claim 1, characterized in that, S5 includes: S51. Input the global context feature map into the multi-head semantic decoding module to perform shared decoding mapping and obtain the shared decoding feature map; S52. In the multi-head semantic decoding module, construct nodule semantic head, cavity semantic head, fiber semantic head and background semantic head respectively. Input the shared decoding feature vector of the shared decoding feature map at the voxel coordinates into the nodule semantic head, cavity semantic head, fiber semantic head and background semantic head respectively to perform category scoring mapping, and obtain nodule category score, cavity category score, fiber category score and background category score respectively; S53. By selecting the category type corresponding to the maximum value among the nodule category score, cavity category score, fiber category score and background category score at the voxel coordinates, the corresponding category type is used as the initial segmentation mask at the voxel coordinates.

7. The lung image segmentation method based on improved SAM2 according to claim 1, characterized in that, S6 includes: S61. Obtain the initial segmentation mask and probability maps of each category, and identify the initial boundary voxel set in the initial segmentation mask; S62. In the boundary refinement module, calculate the HU gradient magnitude for each voxel coordinate for the standardized HU value; S63. For the nodule probability map, cavity probability map, and fiber probability map, calculate the nodule probability gradient magnitude, cavity probability gradient magnitude, and fiber probability gradient magnitude for each voxel coordinate, respectively. S64. Divide the HU gradient vector of the voxel by the HU gradient magnitude to obtain the standardized HU gradient direction unit vector, and calculate the gradient consistency score based on the nodule probability gradient magnitude, cavity probability gradient magnitude and fiber probability gradient magnitude. S65. Traverse all voxel coordinates in the 3D neighborhood. For a neighboring voxel belonging to the initial boundary voxel set, calculate the absolute value of the dot product of the current voxel and the unit vector of the standardized HU gradient direction of the neighboring voxel. Average all the absolute values ​​of the dot products that meet the conditions to obtain the edge continuity score of the voxel coordinates. S66. The gradient consistency score and the edge continuity score are weighted and summed to obtain the boundary enhancement response value in voxel coordinates; S67. Compare the boundary enhancement response value of each initial boundary voxel with the set boundary enhancement threshold to generate a boundary enhancement mask.

8. The lung image segmentation method based on improved SAM2 according to claim 1, characterized in that, S7 includes: S71. In the linkage morphology reconstruction module, based on the category label of the boundary enhancement mask, extract the set of all voxel coordinates labeled as nodule category, the set of all voxel coordinates labeled as cavity category, and the set of all voxel coordinates labeled as fiber category respectively. S72. Based on the extracted voxel coordinate sets of nodule category, cavity category, and fiber category, construct a lesion morphology atlas; S73. For each voxel cluster in the lesion morphology atlas, based on the 26-neighborhood connection rule in three-dimensional space, spatially connected voxels of the same category are automatically clustered into independent voxel clusters. S74. Calculate the morphological consistency score for each voxel cluster and perform morphological consistency screening; S75. After completing the morphological consistency screening, for each voxel in all remaining voxel clusters, re-label it according to the voxel cluster category as the corresponding lesion category, and generate a morphological atlas correction mask.

9. The lung image segmentation method based on improved SAM2 according to claim 7, characterized in that, The morphological consistency screening includes: If a voxel cluster is a nodule category and its morphological consistency score is lower than the nodule category morphological consistency threshold, then the category label of all voxels in the corresponding voxel cluster is set to the background category. If a voxel cluster is a hole category and its morphological consistency score is lower than the morphological consistency threshold for the hole category, then the category label of all voxels in the corresponding voxel cluster is set to the background category. If a voxel cluster belongs to the fiber category and its morphological consistency score is lower than the fiber category morphological consistency threshold, then the category label of all voxels in the corresponding voxel cluster is set to the background category.

Images