CT image segmentation method and electronic device

Abstract

The CT image segmentation method and the electronic device are disclosed. The CT image and the structured meta information corresponding to the CT image are acquired, the CT image is subjected to feature extraction to obtain CT image features of multiple levels, the structured meta information is subjected to feature coding to obtain text features, the text features are subjected to linear transformation to obtain channel gating weights, the CT image features are subjected to channel weighting processing based on the channel gating weights to obtain channel weighted features, the channel weighted features are subjected to feature enhancement to obtain CT image enhanced features, the CT image enhanced features are subjected to decoding to obtain guide features, the guide features and the CT image features of the corresponding level are subjected to feature transformation in the spatial dimension and the channel dimension to obtain joint gating weights, the CT image features are subjected to residual fusion processing based on the joint gating weights to obtain gating output features, and the CT image segmentation is performed based on the gating output features to obtain a CT image segmentation result, thereby improving the segmentation accuracy of the CT image.

Description

Technical Field

[0001] This disclosure relates to the field of image segmentation technology, and in particular to CT image segmentation methods and electronic devices. Background Technology

[0002] In the field of medical imaging, CT images, with their clear representation of the internal structure and tissue density of the human body, have become an important basis for disease diagnosis. Current technologies, in order to balance the accuracy and efficiency of lesion segmentation, typically employ multi-slice stacking to form a multi-channel input, using a two-dimensional network to introduce spatial context for lesion segmentation, and combining attention mechanisms to enhance features of the lesion region to improve segmentation performance. However, existing methods struggle to effectively distinguish between valid contextual information and irrelevant noisy slices, are susceptible to interference from surrounding tissues, leading to unstable segmentation boundaries and missed lesions, thus affecting the accuracy of lesion segmentation. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail in this disclosure. This overview is not intended to limit the scope of the claims.

[0004] This disclosure provides a CT image segmentation method that can improve the accuracy, boundary continuity, and overall robustness of lesion segmentation.

[0005] On one hand, embodiments of this disclosure provide a CT image segmentation method, including: Acquire CT images and their corresponding structured metadata, extract features from the CT images to obtain CT image features at multiple levels, wherein the CT images are composed of multiple two-dimensional CT image slices; The structured metadata is feature-encoded to obtain text features. The text features are then linearly transformed to obtain channel gating weights. Based on the channel gating weights, the CT image features are subjected to channel weighting to obtain channel-weighted features. The channel-weighted features are enhanced to obtain CT image enhancement features. During the feature enhancement process, the feature response intensity of each channel is dynamically adjusted based on the spatial positional relationship of the channel-weighted features. The enhanced features of the CT image are decoded to obtain guiding features. The guiding features and the corresponding CT image features at the same level are transformed in the spatial and channel dimensions to obtain joint gating weights. The CT image features are then subjected to residual fusion processing based on the joint gating weights to obtain gated output features. CT image segmentation is performed based on the gated output features to obtain CT image segmentation results.

[0006] On the other hand, this disclosure also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described CT image segmentation method.

[0007] The embodiments of this disclosure include at least the following beneficial effects: By acquiring CT images and their corresponding structured metadata, feature extraction is performed on the CT images to obtain CT image features at multiple levels. Since CT images are composed of multiple two-dimensional CT image slices, introducing local longitudinal contextual information through multi-neighbor two-dimensional CT image slices can improve the perception capability of small-volume lesions without significantly increasing computational complexity. Feature encoding is performed on the structured metadata to obtain text features. Linear transformation is performed on the text features to obtain channel gating weights. Based on the channel gating weights, channel-weighted processing is performed on the CT image features to obtain channel-weighted features. By converting the structured metadata into channel gating weights and using these channel gating weights as conditional guiding signals to weight the CT image features, the segmentation model can still stably focus on the lesion region even when the lesion contrast is low and the boundaries are blurred, thereby improving the segmentation accuracy of the segmentation model. Feature enhancement is performed on channel-weighted features to obtain CT image enhancement features. During feature enhancement, the feature response intensity of each channel is dynamically adjusted based on the spatial positional relationship of the channel-weighted features, enabling efficient global semantic modeling. Based on this, the CT image enhancement features are decoded to obtain guiding features. Feature transformation is performed on the guiding features and the corresponding CT image features in the spatial and channel dimensions to obtain joint gating weights. Residual fusion processing is performed on the CT image features based on the joint gating weights, which not only suppresses background interference but also enhances the expression of lesion boundaries, resulting in accurate and high-quality gating output features. CT image segmentation is performed based on the gating output features to obtain CT image segmentation results, thereby improving the accuracy, boundary continuity, and overall robustness of lesion segmentation.

[0008] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing this disclosure. Attached Figure Description

[0009] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0010] Figure 1 A schematic diagram of an optional overall structure of the segmentation model provided in an embodiment of this disclosure; Figure 2An optional flowchart of the CT image segmentation method provided in the embodiments of this disclosure; Figure 3 This is a schematic diagram of an optional structure of the metadata encoding module provided in an embodiment of the present disclosure; Figure 4 A schematic diagram of an optional structure of the bottleneck layer provided in an embodiment of this disclosure; Figure 5 A schematic diagram of an optional gated hybrid module provided in an embodiment of this disclosure; Figure 6 A schematic diagram of an optional hybrid attention gating module provided in an embodiment of this disclosure; Figure 7 This is an optional schematic diagram of dual-path parallel processing provided in an embodiment of this disclosure; Figure 8 This is a schematic diagram of an optional overall framework for a CT image segmentation method provided in an embodiment of this disclosure. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0012] It should be noted that in the various specific embodiments of this disclosure, when processing is required based on data related to the characteristics of the target object, such as target object attribute information or a set of attribute information, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. The target object can be a user. In addition, when embodiments of this disclosure require obtaining target object attribute information, separate permission or consent from the target object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of the embodiments of this disclosure be obtained.

[0013] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0014] To facilitate understanding of the technical solutions provided in the embodiments of this disclosure, some key terms used in the embodiments of this disclosure will be explained below: CT images, also known as computed tomography images, are digital cross-sectional images generated based on the attenuation differences of X-rays after penetrating human tissue, combined with computer reconstruction technology. It acquires continuous projection data of a specific cross-section of the human body through synchronous rotation of the X-ray source and detector, then converts it into a grayscale image, presenting tissue density information in the form of a pixel matrix.

[0015] Structured metadata is prior information related to CT images, possessing clear semantics and capable of being structured and represented. It exists in the form of non-image pixels and can provide quantifiable and clearly describable auxiliary support for the segmentation process, providing explicit semantic guidance for lesion segmentation.

[0016] Gating mechanisms are dynamic weighting and filtering mechanisms learned through learning. By generating corresponding gating weights, they adaptively select, suppress, or enhance input features, retaining only key and effective information while filtering out redundant or interfering information, thereby achieving precise control over the feature transfer and fusion process.

[0017] In the field of medical imaging, CT images, with their clear representation of internal human structures and tissue density, have become a crucial basis for disease diagnosis. With the development of deep learning, encoder-decoder structures such as U-Net have gradually become the mainstream method for medical image segmentation. These methods automatically learn multi-scale semantic features from CT images using convolutional neural networks, achieving end-to-end segmentation of lesion regions and effectively replacing traditional manual delineation. However, existing 2D segmentation methods typically use single slices as input, failing to fully utilize the spatial continuity of adjacent slices in CT sequences, and are prone to problems such as lesion breakage, missed detections, and boundary drift. While 3D segmentation methods can explicitly capture the three-dimensional morphology and contextual information of lesions, they suffer from high memory consumption, large computational overhead, and strong dependence on high-quality labeled data, posing a barrier to real-world clinical deployment.

[0018] To balance lesion segmentation accuracy and efficiency, 2.5D segmentation methods have become a preferred compromise. These methods typically employ multi-slice stacking to create a multi-channel input, using a 2D network to introduce spatial context for lesion segmentation, and incorporating attention mechanisms to enhance lesion region features and improve segmentation performance. However, existing 2.5D segmentation methods struggle to effectively distinguish between valid contextual information and irrelevant noisy slices, are susceptible to interference from surrounding tissues, leading to unstable segmentation boundaries and missed lesions. Furthermore, existing 2.5D segmentation methods often focus on learning image pixel features, neglecting the visual guidance of clinical structured prior information during the segmentation process. This makes them unsuitable for complex segmentation scenarios such as small lesions, blurred boundaries, and complex morphologies, thus affecting the accuracy of lesion segmentation.

[0019] Based on this, the present disclosure provides a CT image segmentation method that can improve the accuracy, boundary continuity and overall robustness of lesion segmentation.

[0020] The CT image segmentation method provided in this embodiment is implemented by a segmentation model, referring to... Figure 1 , Figure 1 This is a schematic diagram of an optional overall structure of the segmentation model provided in this embodiment. The segmentation model includes a metadata encoding module, an image encoding module (dashed box area), a hybrid attention gating module, and a bottleneck layer. The metadata encoding module is used to encode the structured metadata; the image encoding module includes multiple image encoding layers for feature extraction from CT images, with each image encoding layer outputting image features at a corresponding level, and features at different levels corresponding to different scales; the hybrid attention gating module is a spatial and channel-coordinated attention gating module used for dynamic filtering of multi-scale (multi-level) features; and the bottleneck layer is used for feature enhancement.

[0021] Reference Figure 2 , Figure 2 This is an optional flowchart of a CT image segmentation method provided in an embodiment of the present disclosure. The CT image segmentation method includes, but is not limited to, the following steps S201 to S205.

[0022] Step S201: Obtain CT images and corresponding structured metadata, extract features from CT images, and obtain CT image features at multiple levels.

[0023] Specifically, CT images are represented as 2.5D medical images, composed of multiple 2D CT image slices. A 2D CT image slice is a complete 2D image extracted from a 3D CT image. Therefore, a CT image is a multi-slice combination centered on a target 2D CT image slice, selecting several adjacent 2D CT image slices before and after it. Structured metadata is clinical information standardized using predefined fields, including basic patient information, diagnostic conclusions, lesion attributes, lesion location, and annotation results. This information is stored in the structured metadata as fields, corresponding one-to-one with the CT images. The structured metadata is obtained by physicians recording and organizing image features during CT image review and lesion annotation, and then exporting it in a case-level file format, specifically CSV format.

[0024] Structured metadata includes numerical metadata and discrete metadata. Numerical metadata describes the spatial distribution characteristics of lesions in in vivo data, while discrete metadata provides coarse-grained structural prior information. Taking T-images of lung adenocarcinoma as an example, numerical metadata includes statistical information such as thickness, initiation layer, maximum layer, and termination layer. Thickness, initiation layer, maximum layer, and termination layer can be stored as numerical fields in numerical metadata. Discrete metadata includes the anatomical location of the lesion (i.e., lesion location in the following text) and the lesion centrality grade. The lesion location and lesion centrality grade can be stored as discrete fields in discrete metadata.

[0025] During training, physicians extract corresponding structured meta-information while reviewing CT images and lesion annotations. The CT images and their corresponding structured meta-information are used as training samples. During inference, structured meta-information can be obtained by parsing similar structured report fields, or physicians can supplement this with patient clinical information, converting the patient clinical information into structured meta-information. This structured meta-information is then used as structured medical prior information to segment the CT images.

[0026] Specifically, CT images and their corresponding structured metadata are acquired. The CT images are then input into an image encoding module for feature extraction. Furthermore, the CT images are downsampled at different scaling ratios to obtain CT image features at multiple levels. For example, ... Figure 1As shown in the dashed box area, multiple 2D CT image slices in the CT image are sequentially downsampled through five image coding layers, outputting CT image slice features with feature dimensions of 24×256×256, 48×128×128, 80×64×64, 176×32×32, and 512×16×16 respectively. It can be understood that the above CT image slice features are extracted for a single 2D CT image slice; therefore, their feature dimensions correspond to the feature output of a single 2D CT image slice. According to the above explanation, the feature dimension of the CT image slice feature is represented as C×H×W, where C is the number of channels, H is the slice height, and W is the slice width. Based on this, when performing feature extraction on a CT image, the feature dimension of the obtained CT image features can be represented as B×C×H×W, where B is the number of 2D CT image slices processed in the batch.

[0027] Step S202: Encode the structured metadata to obtain text features, perform linear transformation on the text features to obtain channel gating weights, and perform channel weighting on the CT image features based on the channel gating weights to obtain channel weighted features.

[0028] In step S202, the feature encoding process of the structured metadata is performed in the metadata encoding module. Channel gating weights are used to perform channel-weighted processing on the CT image features output from the last image encoding layer, corresponding to... Figure 1 The feature dimension of the CT image features output by the last image coding layer is 512×16×16.

[0029] It is understandable that structured metadata includes multiple metadata fields; however, during the acquisition of structured metadata, some metadata fields may be missing. To adapt to situations where structured metadata is incomplete, this disclosure adopts a unified secure mapping strategy for missing metadata fields, enabling the segmentation model to stably degenerate into a learning mode primarily based on CT image features when some structured metadata is lacking.

[0030] In one possible implementation, during the process of feature encoding of structured metadata to obtain text features, and performing linear transformation on the text features to obtain channel gating weights, specifically, nonlinear mapping can be performed on numerical metadata to obtain numerical mapped features, and feature embedding can be performed on discrete metadata to obtain discrete embedded features. The numerical mapped features and discrete embedded features are concatenated along the channel dimension to obtain text features, which are then linearly transformed. The channel gating weights are determined based on the gating activation bias coefficients and the linearly transformed text features. The discrete embedded features include lesion location embedding representations and centrality embedding representations.

[0031] Specifically, refer to Figure 3 , Figure 3 This is a schematic diagram of an optional structure of the metadata encoding module provided in an embodiment of this disclosure, combined with... Figure 3 The feature encoding process of structured metadata is explained. Numerical metadata (NUM) is converted into numerical feature vectors. A multilayer perceptron is used to extract features from these vectors. Nonlinear mapping is applied to the numerical feature vectors through a first and second fully connected layer to obtain continuous numerical semantic representations, resulting in numerical mapped features. The numerical feature vector can be represented as... The mapping process can be represented by the following formula.

[0032] in, It is a numerical mapping feature. It is the ReLU activation function. , These are the weight matrices for the first fully connected layer and the second fully connected layer, respectively. , These are the bias terms for the first and second fully connected layers, respectively. When a numerical field is missing from the numerical metadata, the dimension value corresponding to the missing numerical field in the numerical feature vector is set to 0, thus marking the missing state of that numerical field. Then, through a non-linear mapping operation, the bias introduced by the missing field is automatically learned and absorbed, thereby avoiding unstable interference from the missing field in the subsequent feature fusion process and improving the segmentation performance and robustness of the segmentation model in scenarios with incomplete numerical metadata.

[0033] Next, feature embedding is performed on the discrete metadata. In the embedding layer, the lesion location (POS) and lesion centrality level (CEN) are projected into the continuous semantic space using an embedding matrix to obtain discrete embedded features. The feature embedding process can be represented by the following formula.

[0034] in, The location of the lesion is defined as ranging from 1 to 6. The lesion centrality level is defined as ranging from 1 to 3. The location of the lesion is embedded in the representation. For central embedding representation, Embed a matrix to locate the lesion. The centrality embedding matrix is ​​used. When a discrete field is missing from the discrete metadata or the value of a discrete field exceeds the defined range, the discrete field is uniformly mapped to a preset default embedding index. This allows the segmentation model to treat unknown discrete metadata as a learnable neutral semantic state, thereby ensuring the consistency of the metadata encoding module between the training and inference phases.

[0035] Numerical mapping features, lesion location embeddings, and centrality embeddings are concatenated along the channel dimension. The resulting concatenated features are mapped into a unified semantic vector. Multi-source feature fusion is achieved through linear and normalization layers in a multilayer perceptron to obtain text features. The encoding process of text features can be represented by the following formula.

[0036] in, The fusion weight matrix for multi-source metadata. This is the fusion bias term. This formula enables the segmentation model to fully utilize structural priors when structural metadata is complete, and to naturally revert to a low-weight, low-interference semantic representation when structural metadata is missing.

[0037] Furthermore, to achieve dynamic interaction between CT image features and text semantics, a linear transformation is performed on the text features. The Sigmoid function is then used to apply nonlinear constraints to the linearly transformed text features, resulting in text-constrained features. Channel gating weights are determined based on the gating activation bias coefficients and the text-constrained features. These channel gating weights can be expressed by the following formula.

[0038] Here, 0.35 and 0.65 are both gating activation bias coefficients, which are predefined constants used to limit the gating activation range. For the Sigmoid function, This is the gated linear transformation matrix. By introducing gated activation bias coefficients, the problem of feature oversuppression can be effectively alleviated, thereby improving the robustness and stability of the segmentation model in scenarios with discrete missing metadata.

[0039] After obtaining the channel gating weights, the text features are adaptively weighted by multiplying the CT image features output from the last image coding layer to obtain channel-weighted features. This process can be represented by the following formula.

[0040] in, This indicates a channel-by-channel multiplication operation, such as Figure 1 As shown, This represents the CT image features output from the last image coding layer. This represents the channel-weighted features. This adaptive channel-weighted processing can recalibrate the CT image feature responses based on the semantic information of the case. By strengthening the feature channel responses related to text semantics, it achieves adaptive enhancement and suppression of feature channels, enabling the segmentation model to generate semantically consistent and stable channel-weighted features regardless of whether the structured meta-information is complete or partially missing.

[0041] Step S203: Perform feature enhancement on the channel weighted features to obtain CT image enhancement features.

[0042] In step S203, the feature enhancement process is performed in the bottleneck layer. During the feature enhancement process, the feature response intensity of each channel is dynamically adjusted based on the spatial positional relationship of the channel weighted features.

[0043] In one possible implementation, during the process of enhancing the channel-weighted features to obtain the CT image enhancement features, reference is made to... Figure 4 , Figure 4 This is a schematic diagram of an optional bottleneck layer provided in an embodiment of the present disclosure. The bottleneck layer includes a first gating hybrid module, a non-local self-attention module, a second gating hybrid module, a spatial attention module, and a local cross-attention module, combined with... Figure 4 The feature enhancement process is explained. Specifically, channel-weighted features... The feature dimension can be represented as W×H×C, channel-weighted feature. First, the first gated hybrid module is input, and the feature response intensity of each channel of the channel-weighted features is dynamically adjusted based on the spatial positional relationship of the channel-weighted features to obtain the first intermediate feature. The first intermediate feature The input is processed by a nonlocal self-attention module to obtain the second intermediate feature. The second intermediate feature The input is fed into the second gated hybrid module, which dynamically adjusts the feature response intensity of each channel of the second intermediate feature based on the spatial positional relationship of the second intermediate feature to obtain the third intermediate feature. First, the third intermediate feature The input is processed by the spatial attention module, and then processed by the local cross-attention module to obtain the enhancement features of the CT image. The feature dimension of the output CT image enhancement features is consistent with the feature dimension of its input channel-weighted features. By performing feature enhancement at the bottleneck layer, complementary fusion of channel adjustment, global dependency modeling, salient region enhancement, and cross-layer semantic alignment can be achieved.

[0044] In one possible implementation, during the process of sequentially performing spatial attention processing and local cross-attention processing on the third intermediate feature to obtain the CT image enhancement features, such as... Figure 4 As shown, the third intermediate feature The input spatial attention module performs spatial attention processing to obtain spatial attention features. Spatial attention features A linear transformation is performed to obtain the query vector Q. A linear transformation is then performed on the channel-weighted features to obtain the value vector V and the key vector K. The query vector Q, key vector K, and value vector V are then input into a local cross-attention module for local cross-attention processing to obtain the CT image enhancement features. By using the bottleneck feature (third intermediate feature) as the query vector and the bottleneck layer input feature (channel-weighted feature) as the key and value vectors, local cross-attention processing based on the query vector, key vector, and value vector enables cross-layer interaction within a local window, thereby achieving finer semantic alignment and improving the segmentation accuracy of the segmentation model.

[0045] In one possible implementation, during the process of dynamically adjusting the feature response intensity of each channel of the channel-weighted feature based on the spatial positional relationship of the channel-weighted feature to obtain the first intermediate feature, the channel-weighted feature is processed in dual-path parallel processing. In the first path, the channel-weighted feature is transformed to obtain a channel-weighted map. In the second path, the channel-weighted feature is locally spatially encoded to obtain a local spatial encoding tensor. The channel-weighted map and the local spatial encoding tensor are multiplied element-wise to obtain a hybrid-weighted feature. The hybrid-weighted feature and the channel-weighted feature are subjected to residual fusion processing to obtain the first intermediate feature.

[0046] Specifically, refer to Figure 5 , Figure 5 This is an optional schematic diagram of a gated hybrid module provided in an embodiment of this disclosure. The gated hybrid module performs dual-path parallel processing on the input features. F The feature dimension can be represented as W×H×C. Taking the first gated hybrid module as an example, the gated hybrid module is explained below, with input features... F For channel-weighted features, in the second branch, local spatial encoding is performed on the channel-weighted features through a convolutional layer with a kernel size of 3×3 to extract local spatial texture information, resulting in a local spatial encoding tensor. Its encoding process can be represented by the following formula.

[0047] in, This represents the convolution operation. For convolution kernel, For batch normalization operations, It is the ReLU activation function. The input is a channel-weighted feature. A channel gating mechanism is introduced in the first branch, generating a channel-weighted map through a 1×1 convolution kernel. The values ​​in the channel-weighted map are constrained to the [0,1] interval using the Sigmoid function. The channel-weighted map G can be obtained by the following formula.

[0048] in, This represents a 1×1 convolution kernel. The activation function is sigmoid. This channel gating structure can dynamically adjust the feature response intensity of each channel under the condition that it is related to the spatial location of the channel weighted features, so that the segmentation model can adaptively focus on semantic information at different scales in different spatial regions.

[0049] Next, gated modulation is applied to the local spatial coding tensor and the channel-weighted graph. Element-wise multiplication of the local spatial coding tensor and the channel-weighted graph is then performed to achieve spatial mixing, resulting in mixed-weighted features. This enables weighted control of the feature flow (local spatial coding tensor), and the weighting control process can be represented by the following formula.

[0050] in, This indicates an element-wise multiplication operation. The features are then weighted. Finally, feature transformation is performed on the weighted features. High-level semantic features are extracted from the weighted features through convolutional layers. Residual connections are introduced, and residual fusion processing is performed between the high-level semantic features of the weighted features and the channel-weighted features. The fusion process can be represented by the following formula.

[0051] in, The convolution kernel used for feature transformation. The output of the gated hybrid module is in the first gated hybrid module. This serves as the first intermediate feature. By introducing a residual structure, not only can the vanishing gradient problem during the training process of deep networks be effectively alleviated, but the stability of local details in the multi-layer feature transmission is also guaranteed, thereby improving the training stability of the segmentation model and thus improving the accuracy of the segmentation model in lesion segmentation.

[0052] Based on the above description, in the second gated hybrid module, the input feature is the second intermediate feature, and the second intermediate feature is processed in parallel to obtain the third intermediate feature. The processing process is similar to the above process and will not be repeated here.

[0053] By introducing a gated mixing module into the bottleneck layer, the temporal mixing in the existing gated weighting scheme is transferred from the sequence domain to the spatial mixing of the two-dimensional feature map. The stable information flow structure of "gated weighted flow + residual information flow" in the existing gated weighting scheme is preserved. This allows the gated mixing module to achieve learnable information flow adjustment in the two-dimensional spatial domain through "local spatial mixing - gated weighted flow - residual information flow," enabling efficient global semantic modeling. Furthermore, in the bottleneck layer, the gated mixing module interacts with non-local self-attention modules, spatial attention modules, and local cross-attention modules, enhancing the feature extraction and representation capabilities of the segmentation model at different scales and dimensions, thus providing richer and more discriminative feature representations. The interaction of multiple modules can compensate for the limitations of a single structure in feature modeling, achieving effective fusion of global dependencies, local spatial relationships, and cross-layer information. This improves feature richness and discriminativity while enhancing the robustness and generalization ability of the segmentation model, thereby improving its segmentation performance.

[0054] Step S204: Decode the enhancement features of the CT image to obtain the guiding features. Perform feature transformation on the guiding features and the corresponding CT image features in the spatial and channel dimensions to obtain the joint gating weights. Perform residual fusion processing on the CT image features based on the joint gating weights to obtain the gated output features.

[0055] Specifically, the enhancement features of CT images are decoded by a decoder to obtain different levels of guiding features output by the decoder. These guiding features can be represented as follows: Obtain CT image features output from the image coding layer at the same level as the guiding features. CT image features can represent... , where i represents the i-th image encoding layer. Step S204 is executed in the hybrid attention gating module, which combines the guiding features of the same level. p Features of CT images The input is processed by a hybrid attention gating module to perform feature fusion, resulting in gating output features.

[0056] In one possible implementation, feature transformations are performed on the guiding features and the corresponding CT image features in both spatial and channel dimensions to obtain joint gating weights. Residual fusion processing is then performed on the CT image features based on the joint gating weights to obtain gated output features. During this process, the guiding features and CT image features after channel mapping are added element-wise to obtain initial fused features. Channel attention and spatial attention processing are then performed on the initial fused features to obtain channel weights and spatial masks. Joint gating weights are determined based on the channel weights and spatial masks. Residual fusion processing is then performed on the CT image features based on the joint gating weights to obtain gated output features.

[0057] Specifically, refer to Figure 6 , Figure 6 This is an optional schematic diagram of a hybrid attention gating module provided in an embodiment of this disclosure. First, it uses a convolutional layer with a kernel size of 1×1 and a batch normalization layer to process the guiding features. p and CT imaging features Perform channel mapping to obtain guided mapping features. and CT mapping features After element-wise summing of the guide mapping features and CT mapping features, a nonlinear activation using the ReLU function is applied to obtain the initial fused features. It can be obtained through the following formula, This is the ReLU function.

[0058]

[0059] Next, the initial fused features are processed in parallel using two paths. In one path, channel attention is applied to the initial fused features to obtain channel weights; in the other path, spatial attention is applied to the initial fused features to obtain spatial masks. Joint gating weights are determined based on the channel weights and spatial masks. The CT image features are then weighted based on the joint gating weights. Residual fusion processing is then performed on the CT image features and the weighted CT image features to obtain the gated output features.

[0060] In one possible implementation, during the process of performing channel attention processing and spatial attention processing on the initial fused features respectively to obtain channel weights and spatial masks, referencing Figure 7 , Figure 7 This is an optional schematic diagram of the dual-path parallel processing provided in this embodiment. When performing dual-path parallel processing on the initial fused features, in order to simulate the semantic correlation between multiple slices in the channel dimension, a lightweight one-dimensional convolution is introduced in the right-hand branch to perform channel attention processing on the initial fused features, thereby achieving efficient channel interaction. Specifically, global average pooling is performed on the initial fused features along the spatial dimension to obtain the channel description vector, which can be represented as... Next, after locally modeling the channel description vectors using lightweight one-dimensional convolution, the channel weights are obtained through non-linear activation using the sigmoid function. The channel attention process can be represented by the following formula: ,

[0061] in, This indicates a global average pooling operation. This represents a lightweight one-dimensional convolution. Indicates channel weight, This is the Sigmoid function.

[0062] Simultaneously, spatial attention processing is applied to the initial fused features in the left-hand branch. Specifically, the initial fused features are first subjected to a 1×1 convolution, followed by non-linear activation using the sigmoid function to obtain a spatial mask. The spatial attention processing can be represented by the following formula. For space mask.

[0063]

[0064] In one possible implementation, during the determination of joint gating weights based on channel weights and spatial masks, the channel weights and spatial masks are multiplied element-wise to obtain mixed features. An attention fusion bias term is then obtained, which is added to the mixed features. The result of this addition is then non-linearly activated to obtain the joint gating weights. The attention fusion bias term is a learnable parameter used to adaptively compensate for feature responses from different channels and spatial locations during the attention fusion process.

[0065] Specifically, a spatial-channel gating mechanism is introduced to enable the hybrid attention gating module to achieve collaborative modulation in both spatial and channel dimensions. First, the channel weights and spatial mask are multiplied element-wise, and the salient regions in the feature map corresponding to the initial fused features are spatially weighted to obtain the hybrid features. Next, the attention fusion bias term is obtained, added to the hybrid features, and then non-linearly activated using a sigmoid function to obtain the joint gating weights. Based on the above description, the spatial-channel gating mechanism can be expressed by the following formula:

[0066] in, For attention fusion bias, This represents the element-wise broadcast multiplication operation, where M is the joint gating weight. By introducing a spatial-channel gating mechanism to integrate channel dependency modeling and cross-slice pseudo-depth interaction strategies, the feature transfer process between the encoder and decoder can be adaptively adjusted, improving the effectiveness of feature fusion and feature transfer, thereby enhancing the segmentation accuracy of the segmentation model. Furthermore, by setting an attention fusion bias term in the spatial-channel gating mechanism, the segmentation model's ability to focus on key features can be enhanced, thus improving the stability of feature fusion and its feature representation capability.

[0067] In one possible implementation, during the residual fusion processing of CT image features based on joint gating weights to obtain gated output features, specifically, to ensure the training stability of the segmentation model in the early stages of training, an identity residual gating mechanism is used for residual fusion processing. First, the joint gating weights are multiplied channel-by-channel with the CT image features to obtain the weighted CT image features. Next, residual gating coefficients are obtained; these are learnable parameters, and their initial value is set to 0 during training. The weighted CT image features are modulated based on the residual gating coefficients to obtain a gating transform term. Residual fusion processing is then performed on the gating transform term and the CT image features to obtain the gated output features. Based on the above description, the identity residual gating mechanism can be expressed by the following formula.

[0068] in, For gated output characteristics, For residual gating coefficients, This refers to the gating transformation term. By setting residual gating coefficients in the identity residual gating mechanism, the hybrid attention gating module can maintain an identity mapping in the early stages of training, effectively avoiding gradient oscillations, thereby improving the training effectiveness and stability of the segmentation model, and ultimately improving the segmentation accuracy of the segmentation model.

[0069] Step S205: Perform CT image segmentation based on gated output features to obtain CT image segmentation results.

[0070] Specifically, such as Figure 1 As shown, the segmentation model includes four hybrid attention gating modules. Convolutional processing is performed on the gating output features of different levels from each hybrid attention gating module to obtain output features with feature dimensions of 176×32×32, 80×64×64, 48×128×128, and 24×256×256, respectively. During the training phase, the segmentation model undergoes multi-scale depth supervision. The output feature with a feature dimension of 24×256×256 is used as the main output. Auxiliary head 1 outputs features with a feature dimension of 80×64×64, and auxiliary head 2 outputs features with a feature dimension of 48×128×128. The output features with feature dimensions of 80×64×64 and 48×128×128 are used as auxiliary outputs. Prediction is performed based on the main and auxiliary outputs to obtain the predicted segmentation result. The segmentation model is trained based on the difference between the predicted and actual segmentation results. During the inference phase, CT image segmentation is performed based on the output feature with a feature dimension of 24×256×256 to obtain the CT image segmentation result. By employing a multi-scale deep supervision strategy during the training phase, the segmentation model can improve the accuracy, boundary continuity, and overall robustness of lesion segmentation while maintaining lightweight computation and low resource consumption.

[0071] Reference Figure 8 , Figure 8 This is a schematic diagram of an optional overall framework for the CT image segmentation method provided in this embodiment. The CT image segmentation method provided in this embodiment can be applied to the field of intelligent healthcare, specifically to scenarios such as lesion segmentation for lung adenocarcinoma. The principle of the CT image segmentation method in this embodiment is described in general and complete below: The CT image segmentation method provided in this embodiment is implemented by a segmentation model, which includes a metadata encoding module, an image encoding module, a hybrid attention gating module, and a bottleneck layer. During the training of the segmentation model, the AdamW optimizer is used for training optimization, with a batch size of 1 and an initial learning rate of 1e-4. The collected training data is divided into training, validation, and test sets according to a certain ratio. Each training epoch is tested on the validation set until the classification performance of the validation set test results no longer improves after 20 consecutive training epochs, thus completing the training of the segmentation model. Furthermore, a multi-scale supervision strategy is introduced during training. Multiple auxiliary heads are set to output auxiliary features, which are then used in conjunction with the main features output by the final segmentation model to predict the predicted segmentation result. The segmentation model is then trained based on the difference between the predicted and actual segmentation results.

[0072] After training, CT images and their corresponding structured metadata are acquired. This structured metadata includes both numerical and discrete metadata. The image encoding module extracts features from the CT images, resulting in multi-level CT image features. The metadata encoding module encodes and concatenates the numerical and discrete metadata separately to obtain text features. Furthermore, to achieve dynamic interaction between CT image features and text semantics, a linear transformation is performed on the text features. Nonlinear constraints are then applied to the transformed text features to obtain text-constrained features. Channel gating weights are determined based on the gating activation bias coefficients and the text-constrained features. Adaptive channel weighting is then applied to the CT image features output from the image encoding module based on these channel gating weights to obtain channel-weighted features.

[0073] Next, the channel-weighted features are input into the bottleneck layer for feature enhancement. Within the bottleneck layer, the features sequentially pass through a first gated hybrid module, a non-local self-attention module, a second gated hybrid module, a spatial attention module, and a local cross-attention module to obtain CT image enhancement features. In the first and second gated hybrid modules, the input features undergo dual-path parallel processing. In the first path, channel transformation is performed on the input features to obtain a channel-weighted map. In the second path, local spatial encoding is performed on the channel-weighted features to obtain a local spatial encoding tensor. The local spatial encoding tensor and the channel-weighted map are then gated and modulated. The channel-weighted map and the local spatial encoding tensor are then multiplied element-wise to obtain hybrid-weighted features. Finally, residual fusion processing is performed between the hybrid-weighted features and the input features to obtain the output features.

[0074] Next, in the feature recovery stage, the CT enhancement features are decoded to obtain guiding features. These guiding features and the corresponding CT image features are then input into a hybrid attention gating module for feature fusion. In any hybrid attention gating module, the guiding features and CT image features are feature-mapped and then added element-wise to obtain initial fused features. Channel attention and spatial attention processing are then applied to these initial fused features to obtain channel weights and a spatial mask. A spatial-channel gating mechanism is introduced, multiplying the channel weights and spatial mask element-wise to obtain hybrid features. The attention fusion bias term is added to the hybrid features, and the addition result is non-linearly activated to obtain joint gating weights. Based on these joint gating weights, the CT image features are multiplied channel-wise to obtain weighted CT image features. The weighted CT image features are modulated based on residual gating coefficients to obtain a gating transform term. Residual fusion processing is then performed on the gating transform term and the CT image features to obtain the gated output features.

[0075] Finally, the gated output features of the last hybrid attention gating module are convolved, and CT image segmentation is performed based on the convolution result to obtain the CT image segmentation result.

[0076] In summary, the CT image segmentation method provided in this disclosure introduces local longitudinal contextual information through multi-neighbor slice stacking within a two-dimensional network framework, effectively improving the perception of small-volume lesions without significantly increasing computational complexity. Simultaneously, by converting clinical information into structured meta-information and using this structured meta-information as a conditional guiding signal to modulate CT image features, the segmentation model can stably focus on the lesion region even in complex scenarios with low lesion contrast and blurred boundaries, thereby reducing the false negative rate of the segmentation model. Furthermore, feature enhancement through the interaction of multiple modules in the bottleneck layer effectively integrates global dependencies, local spatial relationships, and cross-layer information. The introduction of a gated hybrid module in the bottleneck layer, which achieves learnable information flow adjustment through a "local spatial mixing - gated weighted flow - residual information flow" approach, enables efficient global semantic modeling. In the feature recovery stage, a hybrid attention gating module is used to dynamically filter cross-layer features, effectively suppressing background interference, enhancing boundary feature expression, and improving segmentation accuracy. Meanwhile, introducing a multi-scale supervision strategy during the training phase can improve the training stability and scale consistency of the segmentation model. This allows the segmentation model to further enhance the accuracy, boundary continuity, and overall robustness of lesion segmentation while maintaining lightweight computation and low resource consumption, laying a good foundation for the clinical application of the CT image segmentation method provided in this embodiment.

[0077] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate to describe embodiments of this disclosure, for example, those that can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0078] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0079] It should be understood that in the description of the embodiments of this disclosure, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0080] In the embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0081] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0083] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0085] The above is a detailed description of the preferred embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A CT image segmentation method, characterized in that, include: Acquire CT images and their corresponding structured metadata, extract features from the CT images to obtain CT image features at multiple levels, wherein the CT images are composed of multiple two-dimensional CT image slices; The structured metadata is feature-encoded to obtain text features. The text features are then linearly transformed to obtain channel gating weights. Based on the channel gating weights, the CT image features are subjected to channel weighting to obtain channel-weighted features. The channel-weighted features are enhanced to obtain CT image enhancement features. During the feature enhancement process, the feature response intensity of each channel is dynamically adjusted based on the spatial positional relationship of the channel-weighted features. The enhanced features of the CT image are decoded to obtain guiding features. The guiding features and the corresponding CT image features at the same level are transformed in the spatial and channel dimensions to obtain joint gating weights. The CT image features are then subjected to residual fusion processing based on the joint gating weights to obtain gated output features. CT image segmentation is performed based on the gated output features to obtain CT image segmentation results.

2. The CT image segmentation method according to claim 1, characterized in that, The structured metadata includes numerical metadata and discrete metadata. The process of feature encoding the structured metadata to obtain text features, and performing a linear transformation on the text features to obtain channel gating weights, includes: The numerical metadata is nonlinearly mapped to obtain numerical mapping features, and the discrete metadata is embedded to obtain discrete embedded features. The numerical mapping features and the discrete embedding features are concatenated along the channel dimension to obtain the text features; The text features are linearly transformed, and the channel gating weights are determined based on the gating activation bias coefficients and the linearly transformed text features.

3. The CT image segmentation method according to claim 1, characterized in that, The process involves performing feature transformations on the guiding features and the corresponding CT image features in both spatial and channel dimensions to obtain joint gating weights. Based on these joint gating weights, residual fusion processing is then performed on the CT image features to obtain gated output features, including: The guiding features and CT image features after channel mapping are added element by element to obtain the initial fusion features; The initial fused features are subjected to channel attention processing and spatial attention processing respectively to obtain channel weights and spatial masks; Based on the channel weights and the spatial mask, a joint gating weight is determined, and the CT image features are subjected to residual fusion processing based on the joint gating weight to obtain gating output features.

4. The CT image segmentation method according to claim 3, characterized in that, The step of performing channel attention processing and spatial attention processing on the initial fused features to obtain channel weights and spatial masks includes: The initial fusion features are subjected to global average pooling along the spatial dimension to obtain channel description vectors. After local modeling of the channel description vectors, nonlinear activation is performed to obtain channel weights. The initial fused features are subjected to convolution processing, and the convolution-processed initial fused features are subjected to nonlinear activation to obtain a spatial mask.

5. The CT image segmentation method according to claim 3, characterized in that, The determination of joint gating weights based on the channel weights and the spatial mask includes: The channel weights and the spatial mask are multiplied element-wise to obtain the hybrid features; Obtain the attention fusion bias term, add the attention fusion bias term to the hybrid feature, and perform nonlinear activation on the addition result to obtain the joint gating weight.

6. The CT image segmentation method according to claim 3, characterized in that, The residual fusion processing of the CT image features based on the joint gating weights to obtain gated output features includes: The joint gating weights are multiplied channel by channel with the CT image features to obtain the CT image weighted features; Obtain residual gating coefficients, modulate the weighted features of the CT image based on the residual gating coefficients to obtain a gating transformation term, and perform residual fusion processing on the gating transformation term and the CT image features to obtain gating output features.

7. The CT image segmentation method according to claim 1, characterized in that, During feature enhancement, the feature response intensity of each channel is dynamically adjusted based on the spatial positional relationship of the channel-weighted features. The feature enhancement of the channel-weighted features to obtain CT image enhancement features includes: Based on the spatial positional relationship of the channel weighted features, the feature response intensity of each channel of the channel weighted features is dynamically adjusted to obtain a first intermediate feature. The first intermediate feature is then subjected to nonlocal self-attention processing to obtain a second intermediate feature. Based on the spatial positional relationship of the second intermediate feature, the feature response intensity of each channel of the second intermediate feature is dynamically adjusted to obtain the third intermediate feature. The third intermediate feature is then subjected to spatial attention processing and local cross-attention processing in sequence to obtain CT image enhancement features.

8. The CT image segmentation method according to claim 7, characterized in that, The dynamic adjustment of the feature response intensity of each channel of the channel-weighted features based on the spatial positional relationship of the channel-weighted features to obtain the first intermediate feature includes: The channel-weighted features are processed in a dual-path parallel manner. In the first path, the channel-weighted features are transformed to obtain a channel-weighted map. In the second path, the channel-weighted features are encoded locally to obtain a local spatial encoding tensor. The channel-weighted map and the local spatial coding tensor are multiplied element-wise to obtain a hybrid weighted feature. The hybrid weighted feature and the channel-weighted feature are then subjected to residual fusion processing to obtain a first intermediate feature.

9. The CT image segmentation method according to claim 7, characterized in that, The process of sequentially performing spatial attention processing and local cross-attention processing on the third intermediate feature to obtain CT image enhancement features includes: Spatial attention processing is performed on the third intermediate feature to obtain spatial attention features; A linear transformation is performed on the spatial attention features to obtain a query vector, and a linear transformation is performed on the channel weighted features to obtain a value vector and a key vector; Local cross-attention processing is performed based on the query vector, the value vector, and the key vector to obtain CT image enhancement features.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the CT image segmentation method according to any one of claims 1 to 9.

Images