Focus segmentation method and system based on large model
By introducing the Canny operator and the edge-guided shared attention fusion module, combined with the two-layer routing attention mechanism and residual downsampling, the problem of balancing global semantics and local boundaries in the segmentation of pneumonia lesions in CT images was solved, achieving higher precision lesion segmentation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-13
AI Technical Summary
Existing technologies for segmenting pneumonia lesions in CT images struggle to simultaneously achieve global semantic consistency and local boundary accuracy, and also have limited receptive fields and lack explicit edge guidance and structural constraint mechanisms.
A lesion segmentation method based on a large model is adopted, which combines an encoder module, a decoder module, and a skip connection module. The Canny operator and an edge-guided shared attention fusion module are introduced. Through a two-layer routing attention mechanism and a residual downsampling module, the collaborative optimization of local detail features and global context information is achieved.
It improves the accuracy and robustness of lesion segmentation results, enhances sensitivity to fuzzy boundaries and small lesion areas, and improves the accuracy of boundary localization and morphological reconstruction.
Smart Images
Figure CN121661346A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically to a lesion segmentation method and system based on a large model. Background Technology
[0002] Computed tomography (CT) imaging is a non-invasive imaging technique widely used in medical diagnosis. It reconstructs the internal structure of the human body using X-ray projection data from different angles, clearly revealing typical features of pneumonia such as ground-glass opacities and consolidation. Therefore, precise and automated segmentation and quantitative analysis of pneumonia lesions in CT images are of irreplaceable value for early diagnosis, disease assessment, and monitoring of treatment efficacy.
[0003] Existing technologies utilize machine learning algorithms such as threshold segmentation and region growing to segment lesions. However, due to the generally blurred edges, complex morphology, and high variability of pneumonia lesions, these machine learning algorithms, while adept at extracting local features, have limited receptive fields, making it difficult to model global contextual information. Furthermore, they suffer from shortcomings in detail capture and training stability. Secondly, lesion segmentation requires a balance between global semantic consistency and local boundary accuracy, but existing methods often employ simple feature concatenation or addition for fusion, lacking explicit edge guidance and structural constraint mechanisms.
[0004] Therefore, there is an urgent need for a method and system that can improve the accuracy and robustness of lesion segmentation results. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides a lesion segmentation method and system based on a large model, aiming to automatically segment lesions in CT images and achieve lesion prediction results with clearer boundaries and more complete structures.
[0006] This invention discloses a lesion segmentation method based on a large model, comprising: The CT image to be segmented is acquired, and preprocessing is performed on the CT image to obtain the target image, which is a two-dimensional image with lesion areas. The target image is input into a pre-trained lesion segmentation model to obtain the lesion segmentation result output by the lesion segmentation model. The lesion segmentation model is a deep network model used to identify and segment lesion regions of the target image. The lesion segmentation model includes an encoder module, a decoder module, and a skip connection module. The skip connection module introduces the Canny operator and the edge-guided shared attention fusion module. The skip connection module is used to fuse the encoded features generated by the encoder module and the decoded features generated by the decoder module to obtain the connection fusion features.
[0007] Preferably, the CT image is in three-dimensional format, and preprocessing is performed on the CT image, including: A cross section of the CT image is captured along a preset spatial axis and a preset fixed interval to convert the CT image into multiple two-dimensional slice images; Multiple two-dimensional slice images are normalized based on a preset image size to obtain multiple target images.
[0008] Preferably, the encoder module includes a first encoder, a second encoder, and a bidirectional feature fusion module; The first encoder is built based on a convolutional neural network and is used to extract local detail features of the target image; The second encoder is a feature encoder based on a two-layer routing attention mechanism, used to extract global top and bottom feature information of the target image; The bidirectional feature fusion module connects the first encoder and the second encoder, and is used to fuse local detail feature information and global top and bottom feature information to obtain coded fusion features.
[0009] Preferably, the bidirectional feature fusion module generates coded fusion features using the following formula: in, Indicates the first Each encoded fusion feature, Indicates the first Each channel fusion module Indicates the second encoder's first Global top-bottom feature information, Indicates the first encoder's first... Detailed feature information, Indicates the first Feature enhancement functions, express SE Attention mechanism This represents the convolution operation. This indicates a normalization operation. This represents the activation function.
[0010] Preferably, the second encoder includes a residual downsampling module, which generates downsampling features of the target image based on wavelet transform, including: Extract input features from the target image; The input features are transformed by Haar wavelet to obtain high-frequency and low-frequency components. The high-frequency components include high-frequency components in the horizontal direction, high-frequency components in the vertical direction, and high-frequency components in the diagonal direction. The high-frequency components and low-frequency components are spliced together along the channel dimension to obtain the splicing feature; Perform a convolution operation on the concatenated features to generate downsampled features.
[0011] Preferably, the process by which the skip connection module executes the encoded features generated by the fusion encoder module and the decoded features generated by the decoder module includes: Edge prior features of the target image are extracted based on the Canny operator, and an edge attention mask is generated based on the edge prior features; Attention weights are calculated based on encoding and decoding features using a shared attention mechanism. These attention weights include channel attention weights and spatial attention weights. Attention weights are used to enhance the encoded features and decoded features respectively, resulting in enhanced encoded features and enhanced decoded features; By fusing encoding features, decoding features, enhanced encoding features, and enhanced decoding features, shared attention features are obtained. An initial prediction mask is generated based on the decoded features, and an inverse attention mask and a boundary attention mask are calculated based on the initial prediction mask. The reverse attention mask, boundary attention mask, and edge attention mask are applied to the shared attention feature to generate multi-path weighted features. Multi-path weighted features are fused to obtain multi-path fused features, and connection fused features are generated based on the multi-path fused features through bidirectional attention operation.
[0012] Preferably, the edge prior features of the target image are extracted based on the Canny operator, including: Perform contrast enhancement processing on the target image to obtain the enhanced image; The enhanced image is converted to a grayscale image, and the grayscale image is smoothed using a Gaussian kernel function; Calculate the gradient magnitude of the smoothed grayscale image, including the gradient magnitude in the horizontal direction and the gradient magnitude in the vertical direction; Non-maximum suppression is applied to the gradient magnitude to obtain the gradient result; Edge prior features are obtained by filtering gradient results using a dual-threshold detection function.
[0013] Preferably, the generation of connection fusion features based on multi-path fusion features through bidirectional attention operations includes: Global average pooling is performed on the multi-path fusion features in the height and width directions respectively to obtain the feature vectors in the height and width directions. The height-direction feature vector and the width-direction feature vector are sequentially concatenated, convolved, and activated to generate a bidirectional attention weight map. The bidirectional attention weight map and the shared attention feature are weighted and fused to generate a connection fusion feature.
[0014] Preferably, the lesion segmentation method based on a large model further includes: Obtain the training dataset, which includes CT image samples labeled with lesion areas; Input the training dataset into the lesion segmentation model and obtain the predicted segmentation results output by the lesion segmentation model; The difference between the predicted segmentation result and the labeled lesion region was calculated using the cross-entropy loss function, and the model parameters of the lesion segmentation model were optimized through backpropagation.
[0015] This invention discloses a lesion segmentation system based on a large model, used to execute a lesion segmentation method based on a large model. The lesion segmentation system based on a large model includes: The image processing module is configured to: acquire the CT image to be segmented, and perform preprocessing on the CT image to obtain the target image, which is a two-dimensional image with lesion regions; The large model prediction module is configured to: input the target image into a pre-trained lesion segmentation model and obtain the lesion segmentation result output by the lesion segmentation model; the lesion segmentation model is a deep network model used to identify and segment lesion regions of the target image. The lesion segmentation model includes an encoder module, a decoder module, and a skip connection module. The skip connection module introduces the Canny operator and the edge-guided shared attention fusion module. The skip connection module is used to fuse the encoded features generated by the encoder module and the decoded features generated by the decoder module to obtain the connection fusion features.
[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention achieves multi-module collaborative optimization in lesion segmentation tasks. By introducing a Canny operator-based and edge-guided shared attention fusion module, the lesion segmentation model can simultaneously consider global semantic perception and fine-grained boundary modeling, enhancing its sensitivity to ambiguous boundaries and small lesion regions. Unlike traditional feature-concatenated skip connections, the skip connection module of this invention's lesion segmentation model combines encoding / decoding features, semantic prediction, and edge prior information based on the Canny operator, achieving collaborative guidance of multi-level structural features and enabling the model to achieve higher accuracy in boundary localization and morphological reconstruction. Furthermore, the model combines the complementary characteristics of residual convolutional neural networks and two-layer routing attention to consider both local details and global dependencies; in conjunction with a wavelet downsampling module, it preserves structural information during dimensionality reduction, further improving the detection capability of multi-scale lesions. Attached Figure Description
[0017] Figure 1 A flowchart illustrating the lesion segmentation method based on a large model provided by this invention; Figure 2This is a schematic diagram of the lesion segmentation system based on a large model provided by the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] The present invention will now be described in further detail with reference to the accompanying drawings.
[0020] like Figure 1 As shown in the figure, this embodiment of the invention provides a lesion segmentation method based on a large model, including the following steps.
[0021] S1. Obtain the CT image to be segmented and perform preprocessing on the CT image to obtain the target image.
[0022] In this embodiment of the invention, CT images are typically in three-dimensional format. To facilitate subsequent processing, CT images need to be preprocessed. Specifically, cross-sections of the CT image are captured along a preset spatial axis and at preset fixed intervals to convert the CT image into multiple two-dimensional slice images. Then, the multiple two-dimensional slice images are normalized based on a preset image size to obtain multiple target images.
[0023] For example, the CT image has dimensions of 512×512×100 (length×width×number of slices), with the preset spatial axis selected as the Z-axis (i.e., the direction perpendicular to the long axis of the human body), and a preset fixed interval of 1 mm. Therefore, by taking a section every 1 mm along the Z-axis, the original 3D CT image can be converted into 100 2D slice images, each with a size of 512×512. Next, to standardize the input size, assuming the preset image size is 256×256, each 512×512 slice image needs to be normalized, for example, by reducing it to 256×256 using methods such as bilinear interpolation, thus obtaining multiple target images.
[0024] S2. Input the target image into the pre-trained lesion segmentation model and obtain the lesion segmentation result output by the lesion segmentation model.
[0025] In this embodiment of the invention, the lesion segmentation model is a deep network model for identifying and segmenting lesion regions in a target image. This model includes an encoder module, a decoder module, and a skip connection module. The encoder module extracts feature information from the target image, the decoder module generates lesion segmentation results based on the extracted feature information, and the skip connection module introduces the Canny operator and an edge-guided shared attention fusion module to fuse the encoded features generated by the encoder module and the decoded features generated by the decoder module, resulting in more accurate connection fusion features.
[0026] Specifically, the encoder module may include a first encoder, a second encoder, and a bidirectional feature fusion module. The first encoder is built based on a convolutional neural network and is used to extract local detail features of the target image. The second encoder is a feature encoder based on a two-layer routing attention mechanism and is used to extract global top-bottom feature information of the target image. The bidirectional feature fusion module connects the first encoder and the second encoder and is used to fuse local detail features and global top-bottom feature information to obtain encoded fused features.
[0027] In pneumonia lesion segmentation, lesion regions at different stages exhibit highly dynamic changes in size and morphology. Relying solely on convolutional neural networks or Vision Transformers can easily lead to incomplete feature extraction and loss of detail. Therefore, this application combines a residual convolutional neural network with a feature encoder based on a two-layer routing attention mechanism in the encoder section to simultaneously acquire local structural features and global semantic information. This design not only enhances the model's ability to express multi-scale features but also effectively reduces computational complexity while maintaining accuracy.
[0028] Since residual convolutional neural networks excel at extracting local details, while feature encoders based on a two-layer routing attention mechanism focus more on modeling global dependencies, a bidirectional feature fusion module is used to achieve interactive fusion of local and global information in order to more fully integrate the information from both. This fusion is then fed back to the first encoder and the second encoder, respectively.
[0029] In this embodiment of the invention, the bidirectional feature fusion module generates coded fusion features using the following formula: in, Indicates the first Each encoded fusion feature, Indicates the first Each channel fusion module Indicates the second encoder's first Global top-bottom feature information, Indicates the first encoder's first... Detailed feature information, Indicates the first Feature enhancement functions, express SE Attention mechanism This represents the convolution operation. This indicates a normalization operation. This represents the activation function.
[0030] In this embodiment of the invention, the second encoder includes a residual downsampling module. The process by which the residual downsampling module generates downsampled features of the target image based on wavelet transform includes: extracting input features of the target image; performing wavelet transform on the input features using Haar wavelets to obtain high-frequency components and low-frequency components, wherein the high-frequency components include horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components; concatenating the high-frequency components and low-frequency components along the channel dimension to obtain concatenated features; and performing a convolution operation on the concatenated features to generate downsampled features. This effectively preserves the structural information of the image and avoids the loss of image details caused by traditional downsampling methods (such as pooling operations).
[0031] In the pneumonia lesion segmentation task, the second encoder of this invention introduces a residual downsampling module (i.e., a wavelet downsampling module) to enhance the model's feature extraction capability. This module divides the input features into high- and low-frequency components through frequency domain decomposition, preserving the overall structure of the lesion while enhancing boundary and texture details, thereby improving the model's ability to recognize small lesions and complex boundaries. The residual downsampling module can be represented by the following formula: in, This means that the input features are divided into components in different directions. Indicates low-frequency components. Represents the high-frequency components in the horizontal direction. Represents the high-frequency components in the vertical direction. Indicates the high-frequency components in the diagonal direction; Indicates splicing characteristics, Y This indicates that feature representation learning is performed on the concatenated features, and the required dimensions are output.
[0032] The encoder provided by this invention combines the advantages of convolutional neural networks in local feature extraction with the ability of a feature encoder based on a two-layer routing attention mechanism in global dependency modeling, achieving accurate perception of lesion regions and global contextual understanding. It can effectively capture the complex texture and irregular contour features of lesions, wherein the feature encoder can be a Vision Transformer.
[0033] In pneumonia lesion segmentation, lesion segmentation models often struggle to balance the global perception of high-level semantic features with the boundary representation of low-level detailed features, leading to blurred lesion boundaries, structural discontinuities, and missing details. Traditional edge-guided attention modules typically enhance boundary perception through encoder features and edge priors extracted from the Laplacian pyramid. However, the Laplacian operator is prone to noise and unstable responses in weak boundary regions, and directly utilizing encoder output lacks cross-layer feature interaction. To further improve the model's perception of lesion boundary regions, this invention integrates explicit edge priors and a channel-space shared attention mechanism into the skip connection module, building upon the shared attention skip connection. A Canny operator-based edge-guided shared attention fusion module is proposed. During the decoding stage, the skip connection module fuses semantic prediction features, skip connection features, and edge information extracted by the Canny operator, achieving synergistic optimization of semantic and structural features. This maintains global semantic consistency while more accurately recovering fine-grained boundaries and texture details of the lesions. This enhances the model's boundary sensitivity and morphological fidelity, particularly demonstrating superior robustness and accuracy in segmenting blurred contours and small lesion regions.
[0034] In this embodiment of the invention, the process by which the skip connection module executes the fusion of encoded features generated by the encoder module and decoded features generated by the decoder module includes: extracting edge prior features of the target image based on the Canny operator, and generating an edge attention mask based on the edge prior features; calculating attention weights based on the encoded features and decoder features through a shared attention mechanism, the attention weights including channel attention weights and spatial attention masks; enhancing the encoded features and decoder features using the attention weights respectively to obtain enhanced encoded features and enhanced decoded features; fusing the encoded features, decoded features, enhanced encoded features, and enhanced decoded features to obtain shared attention features; generating an initial prediction mask based on the decoded features, and calculating a reverse attention mask and a boundary attention mask based on the initial prediction mask; applying the reverse attention mask, boundary attention mask, and edge attention mask to the shared attention features respectively to generate multi-path weighted features; fusing the multi-path weighted features to obtain multi-path fused features, and generating connection fused features based on the multi-path fused features through bidirectional attention operations.
[0035] This invention introduces explicit edge priors and a shared attention mechanism based on the Canny operator and an edge-guided shared attention fusion module, thereby achieving dynamic fusion of low-dimensional encoded features and high-dimensional decoded features.
[0036] Specifically, the edge prior features of the target image are extracted based on the Canny operator, including: performing contrast enhancement processing on the target image to obtain an enhanced image; converting the enhanced image into a grayscale image and smoothing the grayscale image using a Gaussian kernel function; calculating the gradient magnitude of the smoothed grayscale image, including the gradient magnitude in the horizontal direction and the gradient magnitude in the vertical direction; performing non-maximum suppression on the gradient magnitude to obtain the gradient result; and performing edge filtering on the gradient result using a dual threshold detection function to obtain the edge prior features.
[0037] For example, for a target image of 256×256, first, contrast enhancement is performed to obtain an enhanced image. Then, the enhanced image is converted to a grayscale image, and a Gaussian kernel function (such as a 3×3 Gaussian kernel) is used to smooth the grayscale image to obtain a smoothed grayscale image. Next, the gradient magnitude of the smoothed grayscale image is calculated, and edge prior features are obtained through nonmaximum suppression by the Canny operator and double threshold detection.
[0038] In this embodiment of the invention, the process of generating connection fusion features based on multi-path fusion features through bidirectional attention operation includes: performing global average pooling on the multi-path fusion features in the height and width directions respectively to obtain a height-direction feature vector and a width-direction feature vector; sequentially performing concatenation, convolution, and activation processing on the height-direction feature vector and the width-direction feature map to generate a bidirectional attention weight map; and weighting and fusing the bidirectional attention weight map with the shared attention features to generate connection fusion features.
[0039] For example, a multi-path fusion feature map of 128×128×256 is first subjected to global average pooling in the height direction to obtain a 1×128×256 feature map; then, it is subjected to global average pooling in the width direction to obtain a 128×128×256 feature map. Next, these two feature maps are concatenated along the channel dimension to obtain a concatenated feature map of 128×128×512. A convolution operation is then performed on the concatenated feature map, for example, using a 1×1 convolution kernel to obtain a 128×128×256 feature map, which is then activated using the Sigmoid function to generate a bidirectional attention weight map. Finally, the bidirectional attention weight map is weighted and fused with the shared attention feature to obtain the connection fusion feature.
[0040] This invention utilizes edge features extracted by the Canny operator to apply guided weighting to multi-scale semantic features, emphasizing the importance of lesion region boundaries. By combining convolutional spatial attention and channel-space joint attention mechanisms, it simultaneously focuses on global semantic consistency and local edge refinement during the fusion process, thereby significantly improving the boundary clarity and structural continuity of lesion regions.
[0041] Specifically, in this embodiment of the invention, the skip connection module that shares attention weights in the i-th decoding stage can be represented by the following formula: ； ； ； ； ； in, Indicates channel attention weights. Represents spatial attention weights. Represents the coding features, Indicates decoding features, Indicates enhanced coding features, This represents the enhanced decoding features, obtained after attention enhancement. To ensure the integrity of the original feature information, It introduces a shared attention feature that combines the original features and the enhanced features.
[0042] Furthermore, features at the current scale of the decoder are utilized. An initial prediction mask is generated, which is expressed using the following formula: ； The initial prediction mask provides the foreground probability distribution, which is used to generate various attention masks. Based on the initial prediction mask... The back attention and boundary attention maps are calculated using the following formula: ； in, express Sigmoid function, Let represent the Laplacian operator, used to capture high-frequency boundary information in the initial prediction mask. Next, the edge prior features E extracted by the Canny operator are introduced as an explicit structural prior, and the edge attention mask is generated using the following formula. This provides clear boundary guidance for the model: ； ； ； ； ； ； in, I This represents the input target image. This represents the contrast enhancement factor. This represents a brightness offset term used to improve the visibility of lung structures. The standard deviation is expressed as Gaussian kernel, gradient magnitude using L2 Norm, NMS Indicates nonmaximum suppression. Represents a double threshold detection function Furthermore, the reverse attention mask, boundary attention mask, and edge attention mask are applied to the shared attention features using the following formula to obtain multi-path weighted features: ； Next, the three features are fused, concatenated along the channel dimension using the following formula, and local context fusion and channel compression are achieved through 3×3 convolution: ； in, express RELU Activation function BN This indicates batch normalization. The fused features are further input using the following formula to generate a pixel-level attention weight map: ； To further highlight salient regions and suppress irrelevant background, this invention enhances the model's ability to perceive spatial location information by aggregating global information in both the horizontal and vertical directions and assigning weights accordingly. This is expressed by the following formula: ； in, The bidirectional attention operation is represented by the following formula: ； ； 。
[0043] In this embodiment of the invention, the decoder module can progressively upsample based on the coded fusion features to generate a lesion segmentation result with the same size as the target image. During the upsampling process, the decoder module gradually restores the spatial information of the image, making the segmentation result more accurate. The decoder module is used to generate lesion segmentation results based on the coded fusion features. Specifically, the decoder module adopts a U-Net structure and restores the resolution of the coded fusion features to the same resolution as the input image through multi-layer upsampling operations, thereby obtaining the lesion segmentation result.
[0044] For example, a feature map with encoded fusion features of 128×128×256 is upsampled to obtain a feature map of 256×256×128. Then, it is processed by a convolutional layer to obtain a feature map of 256×256×64. Next, it is upsampled to obtain a feature map of 256×256×32. Finally, it is processed by a convolutional layer to obtain a lesion segmentation result of 256×256×1.
[0045] In this embodiment of the invention, the lesion segmentation method based on a large model further includes training the lesion segmentation model using a training dataset. Specifically, a training dataset is obtained, which includes CT image samples labeled with lesion regions; the training dataset is input into the lesion segmentation model to obtain the predicted segmentation result output by the lesion segmentation model; the difference between the predicted segmentation result and the lesion region label is calculated using the cross-entropy loss function, and the model parameters of the lesion segmentation model are optimized through backpropagation.
[0046] Through training, the lesion segmentation model can gradually converge to the optimal state during iterative training. It can simultaneously take into account global semantic perception and fine-grained boundary modeling, enhance the sensitivity to fuzzy boundaries and small lesion regions, and improve the accuracy and robustness of lesion segmentation.
[0047] like Figure 2 As shown, the present invention also provides a lesion segmentation system based on a large model for executing a lesion segmentation method based on a large model. The lesion segmentation system based on a large model includes an image processing module 201 and a large model prediction module 202.
[0048] Specifically, the image processing module 201 is configured to: acquire the CT image to be segmented, and perform preprocessing on the CT image to obtain the target image, which is a two-dimensional image with lesion areas.
[0049] The large model prediction module 202 is configured to: input the target image into the pre-trained lesion segmentation model and obtain the lesion segmentation result output by the lesion segmentation model; the lesion segmentation model is a deep network model used to identify and segment lesion regions of the target image. The lesion segmentation model includes an encoder module, a decoder module and a skip connection module. The skip connection module introduces the Canny operator and the edge-guided shared attention fusion module. The skip connection module is used to fuse the encoded features generated by the encoder module and the decoded features generated by the decoder module to obtain the connection fusion features.
[0050] As can be seen from the above technical solution, this application provides a lesion segmentation method and system based on a large model. The lesion segmentation method includes: acquiring a CT image to be segmented, performing preprocessing on the CT image to obtain a target image, the target image being a two-dimensional image with lesion regions; inputting the target image into a pre-trained lesion segmentation model, and obtaining the lesion segmentation result output by the lesion segmentation model. The lesion segmentation model is a deep network model used to identify and segment lesion regions of the target image. The lesion segmentation model includes an encoder module, a decoder module, and a skip connection module. The skip connection module introduces the Canny operator and an edge-guided shared attention fusion module, aiming to improve the segmentation accuracy and structural consistency of the lesion segmentation model under complex boundary conditions through joint modeling of explicit edge priors and attention mechanisms.
[0051] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A lesion segmentation method based on a large model, characterized in that, include: A CT image to be segmented is acquired, and preprocessing is performed on the CT image to obtain a target image, which is a two-dimensional image with lesion regions. The target image is input into a pre-trained lesion segmentation model to obtain the lesion segmentation result output by the lesion segmentation model. The lesion segmentation model is a deep network model used to identify and segment the lesion region of the target image. The lesion segmentation model includes an encoder module, a decoder module, and a skip connection module. The skip connection module introduces the Canny operator and an edge-guided shared attention fusion module. The skip connection module is used to fuse the encoded features generated by the encoder module and the decoded features generated by the decoder module to obtain the connection fusion features.
2. The lesion segmentation method according to claim 1, characterized in that, The CT image is in three-dimensional format, and the preprocessing of the CT image includes: The CT image is cut along a preset spatial axis and at a preset fixed interval to convert the CT image into multiple two-dimensional slice images; Multiple two-dimensional slice images are normalized in size based on a preset image size to obtain multiple target images.
3. The lesion segmentation method according to claim 1, characterized in that, The encoder module includes a first encoder, a second encoder, and a bidirectional feature fusion module; The first encoder is constructed based on a convolutional neural network and is used to extract local detail feature information of the target image; The second encoder is a feature encoder based on a two-layer routing attention mechanism, used to extract global top and bottom feature information of the target image; The bidirectional feature fusion module connects the first encoder and the second encoder, and is used to fuse the local detail feature information and the global top and bottom feature information to obtain coded fusion features.
4. The lesion segmentation method according to claim 3, characterized in that, The bidirectional feature fusion module generates the encoded fusion feature using the following formula: in, Indicates the first Each encoded fusion feature, Indicates the first Each channel fusion module Indicates the second encoder's first Global top-bottom feature information, Indicates the first encoder's first... Detailed feature information, Indicates the first Feature enhancement functions, express SE Attention mechanism This represents the convolution operation. This indicates a normalization operation. This represents the activation function.
5. The lesion segmentation method according to claim 3, characterized in that, The second encoder includes a residual downsampling module, which generates downsampled features of the target image based on wavelet transform, including: Extract the input features of the target image; The input features are subjected to wavelet transform using Haar wavelets to obtain high-frequency components and low-frequency components. The high-frequency components include high-frequency components in the horizontal direction, high-frequency components in the vertical direction, and high-frequency components in the diagonal direction. The high-frequency component and the low-frequency component are spliced together in the channel dimension to obtain the splicing feature; The concatenated features are subjected to a convolution operation to generate the downsampled features.
6. The lesion segmentation method according to claim 1, characterized in that, The process by which the skip connection module performs the fusion of the encoded features generated by the encoder module and the decoded features generated by the decoder module includes: The edge prior features of the target image are extracted based on the Canny operator, and an edge attention mask is generated based on the edge prior features; An attention weight is calculated based on the encoded features and the decoded features using a shared attention mechanism. The attention weight includes channel attention weight and spatial attention weight. The attention weights are used to enhance the encoded features and the decoded features respectively, resulting in enhanced encoded features and enhanced decoded features; By fusing the encoded features, the decoded features, the enhanced encoded features, and the enhanced decoded features, a shared attention feature is obtained; An initial prediction mask is generated based on the decoded features, and an inverse attention mask and a boundary attention mask are calculated based on the initial prediction mask. The reverse attention mask, the boundary attention mask, and the edge attention mask are applied to the shared attention feature to generate multi-path weighted features; The multi-path weighted features are fused to obtain multi-path fused features, and the connection fused features are generated based on the multi-path fused features through bidirectional attention operation.
7. The lesion segmentation method according to claim 6, characterized in that, The step of extracting the edge prior features of the target image based on the Canny operator includes: Perform contrast enhancement processing on the target image to obtain an enhanced image; The enhanced image is converted into a grayscale image, and the grayscale image is smoothed using a Gaussian kernel function; Calculate the gradient magnitude of the smoothed grayscale image, wherein the gradient magnitude includes the horizontal gradient magnitude and the vertical gradient magnitude; Non-maximum suppression is applied to the gradient magnitude to obtain the gradient result; The gradient results are edge-filtered using a dual-threshold detection function to obtain the edge prior features.
8. The lesion segmentation method according to claim 6, characterized in that, The step of generating the connection fusion feature based on the multipath fusion feature through bidirectional attention operation includes: The multi-path fused features are subjected to global average pooling in the height and width directions respectively to obtain the feature vectors in the height and width directions. The height-direction feature vector and the width-direction feature vector are sequentially concatenated, convolved, and activated to generate a bidirectional attention weight map. The bidirectional attention weight map and the shared attention feature are weighted and fused to generate the connection fusion feature.
9. The lesion segmentation method according to claim 1, characterized in that, Also includes: Obtain a training dataset, which includes CT image samples labeled with lesion regions; The training dataset is input into the lesion segmentation model to obtain the predicted segmentation result output by the lesion segmentation model; The difference between the predicted segmentation result and the labeled lesion region is calculated using the cross-entropy loss function, and the model parameters of the lesion segmentation model are optimized through backpropagation.
10. A lesion segmentation system based on a large model, used to execute the lesion segmentation method based on a large model as described in any one of claims 1 to 9, characterized in that, include: The image processing module is configured to: acquire a CT image to be segmented, and perform preprocessing on the CT image to obtain a target image, wherein the target image is a two-dimensional image with lesion regions; The large model prediction module is configured to: input the target image into a pre-trained lesion segmentation model, and obtain the lesion segmentation result output by the lesion segmentation model; the lesion segmentation model is a deep network model for identifying and segmenting lesion regions of the target image, the lesion segmentation model includes an encoder module, a decoder module, and a skip connection module, the skip connection module introduces the Canny operator and an edge-guided shared attention fusion module, the skip connection module is used to fuse the encoded features generated by the encoder module and the decoded features generated by the decoder module to obtain connection fusion features.