A segmentation method for esophageal CT images based on asymmetric sensing enhancement

Abstract

This invention belongs to the field of image processing technology and provides a method for esophageal CT image segmentation based on asymmetric perception enhancement. The method includes: acquiring an esophageal CT image; inputting the esophageal CT image into an image segmentation model to obtain esophageal cancer lesion segmentation results; the image segmentation model includes an encoder, a decoder, and multiple skip connections connecting corresponding layers of the encoder and decoder; a multi-scale feature aggregation module and an asymmetric anatomical structure recognition module are sequentially arranged on the skip connections connecting the bottom layer of the encoder and the bottom layer of the decoder; the multi-scale feature aggregation module performs multi-scale feature extraction and fusion processing on the features output from the bottom layer of the encoder; the asymmetric anatomical structure recognition module includes a flip unit, a first subtraction unit, a first high-efficiency channel attention module, a first multiplication unit, and a first residual unit connected sequentially. This invention improves lesion localization accuracy, achieving high-efficiency lesion segmentation, high-definition lesion region edges, and low artifact residue.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for segmenting esophageal CT images based on asymmetric sensing enhancement. Background Technology

[0002] CT images, or computed tomography images, are used to describe cancerous lesions in esophageal CT images. The esophagus is an asymmetrical anatomical structure, and cancerous lesions often infiltrate irregularly along the esophageal wall. This results in two main characteristics: firstly, the CT values ​​(grayscale) of normal and diseased tissues are relatively similar, meaning that cancerous tissue does not appear as a distinct mass on CT images; secondly, the morphology of cancerous lesions is extremely irregular and heterogeneous, potentially differing significantly from the shape patterns learned during image segmentation model training. This presents a significant challenge to the accurate and automated segmentation of cancerous lesions in esophageal CT images.

[0003] In relevant medical image segmentation techniques, U-shaped networks are often used to construct image segmentation models to segment cancerous lesion regions in medical images. The U-shaped network uses skip connections to introduce features from each layer of the encoder into the corresponding layer of the decoder. The decoder can recover the original size while also obtaining detailed features of the same scale from the encoder, thus increasing the accuracy of the medical image segmentation results. However, when processing esophageal tissue regions with significant heterogeneity (such as the infiltrative area of ​​esophageal tumor margins), the skip connections between the encoder and decoder layers use a fixed mapping relationship. This limits the model's ability to flexibly capture the multi-scale morphological features of complex structures such as esophageal lesion margins, leading to problems such as blurred edges and artifact residues in the segmentation results, resulting in low quality and reliability of the segmentation results. Summary of the Invention

[0004] This application aims to at least solve the technical problems existing in the prior art and provide a method for esophageal CT image segmentation based on asymmetric sensing enhancement.

[0005] This application provides a method for esophageal CT image segmentation based on asymmetric perception enhancement. The method includes: acquiring an esophageal CT image; inputting the esophageal CT image into an image segmentation model to obtain esophageal cancer lesion segmentation results; wherein, the image segmentation model includes an encoder, a decoder, and multiple skip connection paths connecting corresponding layers of the encoder and decoder. A multi-scale feature aggregation module and an asymmetric anatomical structure recognition module are sequentially arranged on the skip connection paths connecting the bottom layers of the encoder and decoder; the multi-scale feature aggregation module performs multi-scale feature extraction and fusion processing on the features output from the bottom layer of the encoder; the asymmetric anatomical structure recognition module includes a flip unit, a first subtraction unit, a first high-efficiency channel attention module, a first multiplication unit, and a first residual unit connected sequentially; the first subtraction unit performs a subtraction operation on the features output from the multi-scale feature aggregation module and the features output from the flip unit; the first multiplication unit multiplies the features output from the multi-scale feature aggregation module with the weight map output from the first high-efficiency channel attention module; and the first residual unit performs a residual operation on the features output from the multi-scale feature aggregation module and the features output from the first multiplication unit.

[0006] The beneficial technical effects of this application are as follows: Based on an improved U-shaped network, the network structure of the image segmentation model of this application is obtained. A multi-scale feature aggregation module and an asymmetric anatomical structure recognition module are sequentially set on the skip connection path connecting the encoder's bottom layer and the decoder's bottom layer. The features output by the encoder's bottom layer are deep features, which extract high-level semantic information from esophageal CT images but lack a unified representation of multi-scale context and contain noise and redundant information. The asymmetric anatomical structure recognition module cannot effectively capture the complex spatial relationships of asymmetric structures directly from the features output by the encoder's bottom layer. Therefore, a multi-scale feature aggregation module is set before the asymmetric anatomical structure recognition module to perform multi-scale feature extraction. By combining features with aggregated features, a more comprehensive and robust feature representation is generated. The aggregated features include spatial relationship information between cancerous lesions and surrounding tissues (such as the relative position of the esophageal wall and mediastinal structures), and reduce noise and redundant information in the feature map. The asymmetric anatomical structure recognition module is designed based on the asymmetric anatomical structure of the esophagus. On the basis of the features output by the multi-scale feature aggregation module, the flipping unit is used to capture the original bilateral feature differences of the esophagus, which can effectively capture the complex spatial relationship of the asymmetric structure of the esophagus. Then, the first high-efficiency channel attention module is used to optimize the channel weights and select important feature channels to capture the subtle features of esophageal cancerous lesions, improve the lesion localization accuracy, and achieve high-efficiency cancerous lesion segmentation, high-definition lesion edges, and low artifact residue. Attached Figure Description

[0007] Figure 1 This is a flowchart illustrating an esophageal CT image segmentation method based on asymmetric sensing enhancement in a preferred embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of the network structure of an image segmentation model in a preferred embodiment of the present invention;

[0009] Figure 3 This is a schematic diagram of the asymmetric anatomical structure recognition module in a preferred embodiment of the present invention;

[0010] Figure 4 This is a schematic diagram of the network structure of the multi-scale feature aggregation module in a preferred embodiment of the present invention;

[0011] Figure 5 This is a schematic diagram of the high-efficiency channel attention module structure in a preferred embodiment of the present invention;

[0012] Figure 6 This is a schematic diagram of the structure of a multi-hybrid convolution module in a preferred embodiment of the present invention;

[0013] Figure 7 This is a schematic diagram of the structure of the phantom network module in a preferred embodiment of the present invention;

[0014] Figure 8 This is a schematic diagram of the structure of a decoupled fully connected attention module in a preferred embodiment of the present invention;

[0015] Figure 9 This is a schematic diagram of the structure of the first phantom module and the second phantom module in a preferred embodiment of the present invention. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0017] This invention provides a method for esophageal CT image segmentation based on asymmetric sensing enhancement. The execution subject of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, or a computer. In other words, the esophageal CT image segmentation method based on asymmetric sensing enhancement can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0018] In a preferred embodiment, an esophageal CT image segmentation method based on asymmetric sensing enhancement is provided, please see [link to relevant documentation]. Figure 1 The method includes:

[0019] Step S1: Acquire CT images of the esophagus;

[0020] Step S2: Input the esophageal CT image into the image segmentation model to obtain the segmentation result of the esophageal cancer lesion.

[0021] In this embodiment, esophageal CT images refer to computed tomography (CT) images of the esophagus. Specifically, an esophageal CT image sequence is acquired, and then one or more esophageal CT images are extracted from the sequence. An image segmentation model is used to process the extracted two or more esophageal CT images sequentially or in parallel. The executing entity is not limited to reading the esophageal CT image sequence from a CT scanner or computed tomography system via a communication interface, or the executing entity can read the esophageal CT images from a memory storing the esophageal CT image sequence.

[0022] In this embodiment, an image segmentation model is pre-trained. This model is used to automatically segment lesion regions from esophageal CT images. It should be noted that the lesion type can be esophageal cancer, or other asymmetric lesions. The esophageal cancer lesion segmentation result includes the probability that each pixel in the esophageal CT image belongs to the cancerous lesion region. A binary image with the same size as the esophageal CT image is constructed. A probability threshold can be preset. The probability of all pixels in the esophageal CT image belonging to the lesion region is iterated. When the probability is greater than the probability threshold, the pixel corresponding to that pixel in the binary image is set to white; when the probability is not greater than the probability threshold, the pixel corresponding to that pixel in the binary image is set to black. This yields the result as shown below. Figure 2 The binary segmented image output on the right.

[0023] In this embodiment, please see Figure 2 The image segmentation model includes an encoder, a decoder, and multiple skip connection paths connecting the corresponding layers of the encoder and decoder. On the skip connection paths connecting the bottom layer of the encoder and the bottom layer of the decoder, a multi-scale feature aggregation module and an asymmetric anatomical structure recognition module are set sequentially.

[0024] In this embodiment, the encoder employs a hierarchical structure to progressively extract and downsample features from the esophageal CT image, obtaining features rich in semantic information and highly abstract. The decoder, based on the features output from the encoder's bottom layer and the spatial detail information provided by the skip connection paths, performs feature recovery and upsampling layer by layer to restore the spatial dimensions of the esophageal CT image and output the esophageal cancer lesion segmentation result.

[0025] For example, the encoder includes Layer downsampling layer, decoder includes The image segmentation model, following the feature propagation direction, sequentially includes a first downsampling layer, a second downsampling layer, ..., the third downsampling layer. Layer downsampling layer, the first Layer upsampling layer, first Upsampling layer, ..., second upsampling layer, first upsampling layer. At this point, the... Layer downsampling layer, the first The upsampling layers serve as the bottom layer of the encoder and the bottom layer of the decoder, respectively. The first downsampling layer corresponds to the first upsampling layer, and the two can be connected via a skip connection path; the second downsampling layer corresponds to the second upsampling layer, and the two can be connected via a skip connection path; ..., the ... Layer downsampling layer and the first The upsampling layers correspond to each other, and the two can be connected by a skip connection path, which includes a total of A jump connection path, It is a positive integer greater than 1.

[0026] In the example above, each downsampling layer includes cascaded convolutional modules and downsampling units, and the convolutional module includes one or more cascaded convolutional kernels of size [missing information]. The convolutional downsampling units are not limited to pooling; for example, max pooling can have a kernel size of [size missing]. Each upsampling layer includes a cascaded concatenation operation, a convolutional module, and an upsampling unit. The upsampling unit is not limited to deconvolution or interpolation. The concatenation operation is used to concatenate the features output by the downsampling unit of the corresponding downsampling layer introduced by the skip connection path with the features output by the previous upsampling layer. The convolutional modules of the upsampling layer and the downsampling layer can have the same structure. Of the hop connection paths, excluding the hop connection path connecting the encoder's underlying layer and the decoder's underlying layer, the remaining M... Each jump connection path can be a direct jump connection.

[0027] In this embodiment, the multi-scale feature aggregation module performs multi-scale feature extraction and fusion processing on the features output from the encoder's underlying layer. Preferably, please see... Figure 4 The multi-scale feature aggregation module includes:

[0028] The first convolutional layer takes the features output from the encoder's lower layers as input and the kernel size of the first convolutional layer can be [size missing]. The first convolutional layer reduces the number of channels in the features output from the encoder's lower layers, thus reducing the dimensionality. The features output from the first convolutional layer are then fed into the spatial path and the channel path, respectively.

[0029] The spatial path includes a multi-scale convolutional group, a first summing unit, a first pooling layer, a second convolutional layer, a first activation function unit, and a second multiplication unit connected in sequence. The features output by the first convolutional layer are input into the multi-scale convolutional group. The first summing unit is used to sum the convolution results of multiple convolutional kernels of different sizes in the multi-scale convolutional group to achieve feature fusion. The second multiplication unit is used to multiply the features output by the first summing unit with the features output by the first activation function unit.

[0030] The channel path includes a second pooling layer, a fourth convolutional layer, a second activation function unit, a fifth convolutional layer, and a sixth activation function unit connected in sequence; the features output by the first convolutional layer are input into the second pooling layer;

[0031] The feature integration unit includes a second residual unit and a third convolutional layer. The second residual unit performs residual operations on the features output by the second multiplication unit and the features output by the sixth activation function unit. The third convolutional layer performs a convolution operation on the operation result of the second residual unit to obtain a feature map with the same feature size as the feature output of the encoder's bottom layer.

[0032] In this embodiment, the spatial path and the channel path process the features output by the first convolutional layer in parallel.

[0033] Multiscale convolutional groups consist of two or more convolutions with different kernel sizes connected in parallel. For example, see [link to example]. Figure 4 The multi-scale convolutional group consists of three parallel convolutional kernels with sizes of [missing information]. , , The convolution process involves element-wise addition of the feature maps (i.e., the results of multiple convolutions with different kernel sizes in the multi-scale convolutional group) using a first addition unit to fuse features at different scales. Then, a first pooling layer aggregates the spatial features output from the first addition unit, and a second convolutional layer performs convolution processing. A first activation function unit then linearly transforms the features output from the second convolutional layer. The kernel size of the second convolutional layer can be [missing information]. The activation function of the first activation function unit is a sigmoid activation function. The first pooling layer is not limited to performing max pooling or average pooling.

[0034] The features output from the encoder's bottom layer are input into the second pooling layer of the channel path. See the channel path for details. Figure 4 The second pooling layer is used to compress the spatial dimension of the features output from the encoder's bottom layer to... The global information is extracted. The output features of the second pooling layer are then processed sequentially through the fourth convolutional layer, the second activation function unit, the fifth convolutional layer, and the sixth activation function unit. The kernel size of the fourth and fifth convolutional layers can be [missing information]. The activation function of the second activation unit is the rectified linear unit function, i.e., the ReLU activation function. The kernel size of the third convolutional layer can be... The activation function of the sixth activation function unit is the sigmoid activation function.

[0035] This implementation uses a multi-scale feature aggregation module to fuse features at different scales, thereby improving the image segmentation model's ability to perceive multiple scales, especially complex morphologies such as lesion edges, and reducing segmentation discontinuities and missed detections.

[0036] In this embodiment, please see Figure 3 The asymmetric anatomical structure recognition module includes a flipping unit, a first subtraction unit, a first high-efficiency channel attention module, a first multiplication unit, and a first residual unit connected in sequence. The first subtraction unit performs a subtraction operation on the features output by the multi-scale feature aggregation module and the features output by the flipping unit. The first multiplication unit is used to multiply the features output by the multi-scale feature aggregation module and the weight map output by the first high-efficiency channel attention module. The first residual unit is used to perform a residual operation on the features output by the multi-scale feature aggregation module and the features output by the first multiplication unit.

[0037] Specifically, the features output by the multi-scale feature aggregation module are: The flip unit flips horizontally left and right. After flipping, the horizontal and horizontal flipping features are obtained. Using the first subtraction unit to... and Perform element-wise subtraction to obtain the original two-sided feature differences. The first efficient channel attention module addresses the differences in original bilateral features. The process is performed to obtain a weighted map of feature differences. , This represents the efficient channel attention processing operation of the first efficient channel attention module. Then, the features output by the multi-scale feature aggregation module are processed by the first multiplication unit. Weight graph with the output of the first high-efficiency channel attention module Element-wise multiplication yields the features Finally, the features output by the multi-scale feature aggregation module are processed using the first residual unit. Features of the output of the first multiplication unit Perform residual operations, specifically element-wise addition, to obtain the output features of the asymmetric anatomical structure recognition module, representing... for:

[0038] ;

[0039] This represents the asymmetric anatomical structure recognition module. The asymmetric anatomical structure recognition module compares the features output by the multi-scale feature aggregation module. The left and right sides are crucial steps in visually capturing subtle esophageal lesions (such as cancerous lesions). Modeling the differences between the two sides of an esophageal CT image using flip units and the first subtraction unit enhances the network's attention to these regions. The lower layers of the encoder are deeper and typically have larger receptive fields; smaller regions in the deeper feature maps correspond to larger regions in the original CT image. In this case, the features of symmetrical left and right regions in these deeper feature maps are more likely to originate from the corresponding anatomical regions in the original image. Therefore, the left and right regions can be directly used to replace the asymmetrical anatomical regions in the original esophageal CT image. The asymmetrical anatomical structure recognition module is designed to specifically extract features from asymmetrical anatomical structures such as esophageal cancer lesions, improving segmentation performance.

[0040] This implementation sequentially sets up a multi-scale feature aggregation module and an asymmetric anatomical structure recognition module along the skip connection path connecting the encoder's bottom layer and the decoder's bottom layer. This achieves a hierarchical optimization principle of feature processing from coarse to fine and from general to specific, maximizing module efficiency. The multi-scale feature aggregation module is located at the bottom layer of the U-shaped network structure. By aggregating deep semantic features at multiple scales, such as organ shape and lesion region, it significantly reduces noise and redundant information in the feature map, generating a more comprehensive and robust feature representation. The aggregated features include spatial correlation information between the lesion and surrounding tissues (such as the relative position of the esophageal wall and mediastinal structure). The asymmetric anatomical structure recognition module can directly learn anatomical constraints (such as esophageal orientation and lumen morphology) based on this, improving the accuracy of lesion localization.

[0041] In this embodiment, preferably, please see Figure 5 The first high-efficiency channel attention module includes:

[0042] The third pooling layer performs global average pooling on the features input to the first efficient channel attention module to obtain compressed features. Specifically, the third pooling layer is preferably a global average pooling layer, which compresses the feature map of each channel of the features input to the first efficient channel attention module into a scalar. Convert to compressed features , This represents the number of channels of the features input to the first efficient channel attention module. , These are the height and width of the features input to the first efficient channel attention module. This achieves the fusion of global contextual information.

[0043] A one-dimensional convolutional layer with learnable kernel size processes compressed features to obtain adaptive channel attention weights. The one-dimensional convolutional layer uses an adaptive kernel to model the channel features in the compressed features, generating channel attention weights. This is equivalent to introducing a learnable one-dimensional convolutional kernel to model global contextual information; this one-dimensional convolutional kernel operates in the channel dimension, capturing the dependencies between local channels.

[0044] The third activation function unit processes the channel attention weights using an activation function to obtain normalized channel attention weights. The activation function of the third activation function unit is a sigmoid activation function, which maps the channel attention weights between (0, 1).

[0045] In this embodiment, the first efficient channel attention module dynamically adjusts the importance of each channel through an adaptive channel attention mechanism. Compared with traditional channel attention mechanisms, the first efficient channel attention module significantly reduces computation and parameter count by introducing an adaptive convolutional kernel size, while maintaining efficient feature extraction capabilities. This first efficient channel attention module enhances the expressive power of important feature channels.

[0046] In a preferred embodiment, please see Figure 2 The encoder includes sequentially connected The layer consists of a downsampling layer and an encoder bottom layer. The decoder includes a decoder bottom layer connected in sequence, and... One-to-one correspondence between layer downsampling layers The system consists of an upsampling layer and an eighth convolutional layer. The eighth convolutional layer performs convolution processing on the features output from the first upsampling layer. A positive integer greater than 1; from level 1 to level 2. Each downsampling layer includes cascaded convolutional modules and downsampling units. It is a positive integer, and 1 , from the 1st floor to the Each upsampling layer includes a cascaded second summing unit, a convolutional module, and an upsampling unit. Each jump connection path connects the downsampling unit of the corresponding downsampling layer to the second summing unit of the corresponding upsampling layer;

[0047] No. layer to the first Each downsampling layer includes cascaded multi-hybrid convolutional modules and downsampling units; layer to the first Each upsampling layer includes a cascaded second summing unit, a multi-hybrid convolution module, and an upsampling unit;

[0048] The encoder's underlying layer and the decoder's underlying layer each include a multi-hybrid convolutional module. A jump connection path, that is, a jump connection path connecting the encoder bottom layer and the decoder bottom layer, connecting the output of the multi-hybrid convolutional module at the encoder bottom layer and the input of the multi-hybrid convolutional module at the decoder bottom layer.

[0049] In this embodiment, preferably, It is 3. The value is 5. For the overall network structure of the image segmentation model at this point, please refer to [link / reference]. Figure 2 The convolutional modules of the upsampling layer and the downsampling layer have the same structure. Each convolutional module consists of one or more cascaded convolutional kernels of a size of [missing information]. The convolution is performed with downsampling units, such as max pooling, and the kernel size can be [size missing]. The upsampling unit is either deconvolution or interpolation. See below. Figure 2 The second summing unit in the upsampling layer is used to add the features imported by the skip connection path corresponding to the upsampling layer to the features output by the upsampling unit in the previous upsampling layer element-wise.

[0050] In this embodiment, to enable the image segmentation model to effectively focus on important feature channels while balancing image segmentation performance and computational complexity, preferably, please see... Figure 6 The multi-hybrid convolutional module includes a parallel visual Mamba convolution, a phantom network module, a third summing unit, and a second efficient channel attention module. Features input to the multi-hybrid convolutional module are simultaneously fed into the parallel visual Mamba convolution and the phantom network module. The third summing unit adds the features output by the parallel visual Mamba convolution and the phantom network module, and the output of the third summing unit is fed into the second efficient channel attention module. The second efficient channel attention module has the same structure as the first efficient channel attention module; please see [link to documentation]. Figure 5 The structural description of the first high-efficiency channel attention module mentioned above will not be repeated here.

[0051] In this embodiment, the parallel visual Mamba convolution, or parallel visual Mamba convolution module for short (PVM), is an existing technology. Refer to the PVM Layer structure disclosed in Chinese Patent Publication No. CN119579627A, or the parallel Vision Mamba (PVM) layer disclosed in Chinese Patent Publication No. CN120931920A; further details are omitted here. The parallel visual Mamba convolution segments the features input from the multi-hybrid convolution module into multiple sub-features, processes each sub-feature through Mamba, and then merges the outputs. This minimizes parameters while maintaining high accuracy and keeping the number of channels constant. Furthermore, running the phantom network module in parallel with the parallel visual Mamba convolution captures long-distance dependencies between pixels at different spatial locations, simultaneously aggregating local and long-distance information, thereby enhancing the expressive power of the features. The second efficient channel attention module automatically selects between the two branches (parallel visual mamba convolution and phantom network module) by adaptively adjusting channel weights, enabling the network to focus more effectively on important feature channels, thereby improving the performance of the image segmentation model.

[0052] In this embodiment, the Phantom Network module can be the existing Phantom Network version 2, i.e., GhostNetV2. Preferably, please see... Figure 7 The Phantom Network module includes:

[0053] The first parallel branch includes a cascaded first phantom module and a first normalization unit;

[0054] The second parallel branch includes a cascaded decoupled fully connected attention module and a fourth activation function unit; the features of the input phantom network module are simultaneously input to the first phantom module and the decoupled fully connected attention module; the fourth activation function unit can be a rectified linear unit function, i.e., the ReLU activation function;

[0055] The fifth multiplication unit multiplies the features output by the first normalization unit and the features output by the fourth activation function unit. Specifically, the multiplication is an element-wise multiplication.

[0056] The second phantom module processes the features output by the fifth multiplication unit;

[0057] The fourth normalization unit normalizes the features output by the second phantom module.

[0058] The first concatenation unit is used to concatenate the features input to the phantom network module and the features output by the fourth normalization unit. Specifically, the concatenation operation employs a concatenation function from deep learning techniques. The function concatenates the features of the input phantom network module and the features output by the fourth normalization unit along the channel dimension.

[0059] In this embodiment, by decoupling the fully connected attention module and the first phantom module in parallel, and using the fifth multiplication unit to multiply and fuse the output features of the second parallel branch and the first parallel branch, feature expansion and enhancement are achieved. The enhanced features are input into the second phantom module and the fourth normalization unit for processing. Finally, the output features of the phantom network module are obtained using the first concatenation unit. The above processing can capture the long-distance dependence between pixels at different spatial locations, thereby enhancing the expressive power of the image segmentation model.

[0060] In this embodiment, and more preferably, please see... Figure 8 The decoupled fully connected attention module includes a downsampling processing unit, a sixth convolutional layer, a second normalization unit, a horizontal fully convolutional layer, a third normalization unit, a vertical fully convolutional layer, and a fifth activation function unit connected in sequence.

[0061] In this embodiment, the downsampling processing unit is not limited to performing max pooling on the features of the input phantom network module. The kernel size of the sixth convolutional layer can be... The shape of the convolution kernel for horizontal full convolution is... , The width represents the convolution width, which can be 3 or 5. Therefore, horizontal full convolution refers to the convolution kernel sliding along the horizontal direction (width direction) on the input feature map, focusing on the relationships between features in the same row (same horizontal line) and capturing horizontal patterns. Correspondingly, the kernel shape of vertical full convolution is... , The height of the convolution can be 3 or 5. As can be seen, vertical full convolution refers to the convolution kernel sliding along the vertical direction (height direction) on the input feature map, focusing on the relationships between features in the same column (same vertical line) and capturing patterns in the height direction. The activation function of the fifth activation function unit is a sigmoid activation function. The purpose of decoupling the fully connected attention module is to cooperate with the first phantom module, gathering pixels at different locations along the horizontal and vertical directions respectively to capture long-distance spatial dependencies.

[0062] In this embodiment, and more preferably, both the first phantom module and the second phantom module adopt the following structure, please see... Figure 9 ,include:

[0063] The seventh convolutional layer processes the features of the input first phantom module or second phantom module. Convolutional processing is used to obtain the first part of the features; the seventh convolutional layer performs pointwise convolution operations.

[0064] Depthwise separable convolutional layers are used to process the first part of the features to obtain the second part of the features;

[0065] The second concatenation unit concatenates the first and second part features to obtain the features output by the first or second phantom module. Specifically, the concatenation function (concat function) in deep learning technology is used to concatenate the first and second part features along the channel dimension.

[0066] In this embodiment, the first phantom module and the second phantom module can generate more feature maps using fewer parameters. Given the features of the first phantom module or the second phantom module as input, , Represents the set of real numbers. , , They represent Height, width, and number of channels. Please see... Figure 9 The first or second phantom module divides the output channel into two parts: a first part of features and a second part of features. To minimize computational overhead, the number of output feature maps is strictly controlled; therefore, the input features... The first part of the features is obtained after point-by-point convolution processing in the seventh convolutional layer. , , This indicates the pointwise convolution operation of the seventh convolutional layer. , express The number of channels. The second part of the features is... The second splicing unit splices the first part of the features and the second part of the features to obtain the features output by the first phantom module or the second phantom module. , , express The number of channels. Compared with ordinary convolutional modules, the first and second phantom modules reduce the number of parameters and computational cost while having the same number of input and output feature maps.

[0067] In a preferred embodiment, please see Figure 2 As shown, the front Each skip connection path is sequentially connected to a spatial attention bridging module (SAB) and a channel attention bridging module (CAB). Both the SAB and CAB are existing attention modules. References can be made to the SAB and CAB in Chinese Patent Publication No. CN120765937A, CN119723101A, CN119723101A, or CN119810117A. Further details are omitted here. By setting the spatial attention bridging module and channel attention bridging module on the skip connection path, the convergence ability of the image segmentation model can be effectively improved, the sensitivity to lesions enhanced, edge clarity significantly improved, and artifact residue reduced.

[0068] When the lesion type is esophageal cancer, the training process of the image segmentation model provided by this invention is described below, including:

[0069] Step 1: Prepare the esophageal cancer lesion dataset. This dataset includes multiple esophageal CT images and a corresponding mask for each image. The CT images include those showing cancerous lesions and those not showing cancerous lesions. Each esophageal CT image and its corresponding mask are the same size, and the pixel positions in the mask correspond to those in the CT image. The value of each pixel in the mask represents whether the pixel at that location in the corresponding esophageal CT image belongs to a cancerous lesion region; 1 indicates a cancerous lesion region, and 0 indicates no cancerous lesion region. The mask can be obtained based on expert segmentation markings of cancerous regions on the esophageal CT images.

[0070] Step 2, construct an asymmetric perceptual enhancement deep learning segmentation network, i.e. Figure 2 The network structure of the image segmentation model shown above.

[0071] Step 3: Divide the esophageal cancer lesion dataset into training, testing, and validation sets according to a preset ratio. The preset ratio is not limited to 8:1:1.

[0072] Step 4: Train the asymmetric sensing augmented deep learning segmentation network using the training set. During training, calculate the loss function value based on the esophageal cancer lesion segmentation results of the esophageal CT image output by the asymmetric sensing augmented deep learning segmentation network and the corresponding mask of the esophageal CT image. Update the network parameters of the asymmetric sensing augmented deep learning segmentation network using stochastic gradient descent (SGD) based on the loss function value. When the training stopping condition is met, stop training and save a set of network parameter values ​​of the asymmetric sensing augmented deep learning segmentation network at which the loss function value is minimized. Load the saved network parameter values ​​into the asymmetric sensing augmented deep learning segmentation network to obtain the trained image segmentation model. The training stopping condition is not limited to reaching the maximum preset number of training iterations or the loss function value being less than a preset loss threshold.

[0073] Step 5: Test and validate the trained image segmentation model using the test set and validation set respectively. If the test and validation pass, the image segmentation model that passes the test and validation is the final image segmentation model. If the test or validation fails, use deep learning model parameter tuning methods such as changing the learning rate and changing the training optimizer, and then return to execute steps 4 and 5.

[0074] In the above training process, the step of calculating the loss function value based on the esophageal cancer lesion segmentation result of the esophageal CT image output by the asymmetric perception enhancement deep learning segmentation network and the corresponding mask of the esophageal CT image is as follows: The calculation formula is:

[0075] ;

[0076] in, , These are the first and second weight parameters, respectively, both greater than 0 and less than 1. .

[0077] Represents cross-entropy loss:

[0078] ;

[0079] in, This represents the total number of pixels in the esophageal CT image and mask. Indicates the index of a pixel in an esophageal CT image and mask; Indicating the first esophageal CT image The label value of each pixel in the mask corresponding to the esophageal CT image is either 1 or 0. 1 means that the pixel belongs to the cancerous lesion area, and 0 means that the pixel does not belong to the cancerous lesion area. The segmentation result of esophageal cancer lesions in esophageal CT images output by an asymmetric perception-enhanced deep learning segmentation network represents the first segment. The probability that a pixel belongs to a cancerous lesion area.

[0080] Indicates the loss in the overlapping region:

[0081] ;

[0082] in, This represents the smoothing term, and its values ​​can be: , Represents the natural constant.

[0083] The above loss function Through cross-entropy loss Provides stable gradient flow and good convergence, with overlapping region loss. It addresses the impact of class imbalance and directly optimizes segmentation quality.

[0084] Based on the trained image segmentation model, comparative experiments were conducted on the test set. The performance of the classic basic segmentation model UNet, the attention-based segmentation model Attention UNet, and the lightweight segmentation model Ultralight-VMUNet were compared. The results are shown in Table 1. The image segmentation model AAES-UNet, obtained by utilizing an asymmetric perception-enhanced deep learning segmentation network, achieves better results in DSC (%), IOU (%), SEN (%), and 95HD (mm) metrics, thus better segmenting esophageal cancer lesions. DSC (%) represents the Dice similarity coefficient, used to measure the degree of overlap between two regions. IOU (%) represents the intersection-union ratio, the ratio of the intersection to the union of the predicted and ground truth regions. SEN (%) represents sensitivity, also known as recall or true positive rate. 95HD (mm) represents the 95% Hausdorff distance, measuring the maximum mismatch between two contour boundaries.

[0085] Table 1. Comparison of intelligent segmentation results of different network models for esophageal cancer lesions

[0086]

[0087] It should be noted that when the lesion type is esophageal ulcer or other types of lesions, the training method of the image segmentation model can refer to the above training process, and will not be repeated here.

[0088] The esophageal CT image segmentation method based on asymmetric sensing enhancement provided in this invention innovatively constructs an asymmetric sensing enhancement deep learning segmentation network, which has the following technical advantages:

[0089] 1. A specially designed asymmetric anatomical structure recognition module is used to capture the features of asymmetric esophageal anatomy and accurately locate lesion areas. This module, connected in series with the multi-scale feature aggregation module at the bottom layer of the U-shaped network, forms a dual-module collaborative mechanism. This mechanism utilizes multi-scale information and enhances the recognition ability of irregularly shaped lesions (such as esophageal cancer), significantly improving the segmentation accuracy of image segmentation models for complex and heterogeneous lesion edges, reducing artifact residue, and optimizing lesion localization accuracy. Because the bottom-level features themselves contain the richest semantic information, their feature expression ability is stronger after multi-scale fusion. At this point, introducing prior knowledge-guided structure recognition can more effectively locate lesions.

[0090] 2. Multi-Hybrid Convolutional Module: In deep feature extraction, this module innovatively integrates parallel visual Mamba convolution and phantom network modules for parallel use. The parallel visual Mamba convolution utilizes the Mamba structure to efficiently model long-range dependencies and global context. The phantom network module, by introducing a decoupled fully connected attention module, effectively enhances the network's ability to capture long-range spatial location dependencies, enabling the image segmentation model to simultaneously aggregate local and long-range information, thus improving feature expressiveness. The outputs of both modules are summed and fed into a second, highly efficient attention module to dynamically select important feature channels for use in the deeper layers (layers 4-6) of the image segmentation model. This replaces traditional convolution for deep feature extraction, enhancing the ability to focus on key features through parallel branching and attention mechanisms, balancing feature richness with computational efficiency.

[0091] 3. The image segmentation model employs an asymmetric perception-enhanced deep learning segmentation network: The backbone architecture uses a six-layer U-shaped network variant, including an encoder (shallow convolutional modules + deep multi-hybrid convolutional modules), a decoder (symmetric structure), and skip connection paths (channel attention bridging modules + spatial attention bridging modules). The network's bottom layer uses a multi-scale feature aggregation module connected in series with an asymmetric anatomical structure recognition module, fusing global context and asymmetric structural information. The multi-hybrid convolutional modules enhance deep feature representation, while the asymmetric anatomical structure recognition module optimizes anatomical structure recognition. These two modules work collaboratively through the U-shaped network framework, achieving a balance between high-precision lesion segmentation, high-resolution lesion region edges and low artifact retention, and low artifact retention and high computational efficiency.

[0092] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0093] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for esophageal CT image segmentation based on asymmetric perception enhancement, characterized in that, The method comprises: obtaining an esophagus CT image; inputting the esophagus CT image into an image segmentation model to obtain an esophagus cancer lesion segmentation result; The image segmentation model comprises an encoder, a decoder, a plurality of jump connection paths connecting corresponding layers of the encoder and the decoder, and a multi-scale feature aggregation module and an asymmetric anatomical structure recognition module arranged in sequence on the jump connection path connecting the bottom layer of the encoder and the bottom layer of the decoder; The multi-scale feature aggregation module performs multi-scale feature extraction and fusion processing on the features output by the bottom layer of the encoder; wherein the multi-scale feature aggregation module comprises: a first convolutional layer; a spatial path comprising a multi-scale convolutional group, a first addition unit, a first pooling layer, a second convolutional layer, a first activation function unit and a second multiplication unit connected in sequence, the features output by the first convolutional layer being input into the multi-scale convolutional group, the first addition unit being used for summing the convolution results of a plurality of convolution kernels of different sizes in the multi-scale convolutional group to realize feature fusion, and the second multiplication unit being used for multiplying the features output by the first addition unit and the features output by the first activation function unit; a channel path comprising a second pooling layer, a fourth convolutional layer, a second activation function unit, a fifth convolutional layer and a sixth activation function unit connected in sequence; the features output by the first convolutional layer being input into the second pooling layer; a feature integration unit comprising a second residual unit and a third convolutional layer, the second residual unit performing residual operation on the features output by the second multiplication unit and the features output by the sixth activation function unit, and the third convolutional layer performing convolution operation on the operation result of the second residual unit to obtain a feature map with the same size as the features output by the bottom layer of the encoder; The asymmetric anatomical structure recognition module comprises a flip unit, a first subtraction unit, a first efficient channel attention module, a first multiplication unit and a first residual unit connected in sequence; the first subtraction unit performs subtraction operation on the features output by the multi-scale feature aggregation module and the features output by the flip unit, the first multiplication unit is used for multiplying the features output by the multi-scale feature aggregation module and the weight map output by the first efficient channel attention module, and the first residual unit is used for performing residual operation on the features output by the multi-scale feature aggregation module and the features output by the first multiplication unit; The encoder comprises, which are connected in sequence a layer down-sampling layer and an encoder bottom layer, and the decoder comprises, which are connected in sequence a decoder bottom layer, a layer up-sampling layer corresponding to the layer down-sampling layer of the encoder, a layer down-sampling layer and an encoder bottom layer, and the decoder comprises, which are connected in sequence a decoder bottom layer, a layer up-sampling layer corresponding to the layer down-sampling layer of the encoder, a layer up-sampling layer, an eighth convolutional layer, is a positive integer greater than 1; the first layer to the Each of the down-sampling layers includes a cascaded convolution module and a down-sampling unit, is a positive integer, and 1 the first layer to the Each of the up-sampling layers includes a cascaded second adding unit, a convolution module and an up-sampling unit, wherein the first adding unit is configured to add a feature map output by the convolution module and a feature map output by the up-sampling unit. The plurality of jump connection paths are respectively connected between the down-sampling units of the corresponding down-sampling layers and the second adding units of the corresponding up-sampling layers. No. layer to the first Each downsampling layer includes cascaded multi-hybrid convolutional modules and downsampling units; layer to the first Each upsampling layer includes a cascaded second summing unit, a multi-hybrid convolution module, and an upsampling unit; The bottom layer of the encoder and the bottom layer of the decoder each comprise a multi-mixed convolution module; The eighth convolutional layer performs convolution processing on the features output by the first layer of the up-sampling layer.

2. The method of claim 1, wherein the asymmetry-awareness based esophageal CT image segmentation method is characterized by, The first efficient channel attention module comprises: a third pooling layer performing global average pooling processing on the features input into the first efficient channel attention module to obtain compressed features; a one-dimensional convolutional layer with learnable convolution kernel size, which processes the compressed features to obtain adaptive channel attention weights; a third activation function unit which performs activation function processing on the channel attention weights to obtain normalized channel attention weights. 3.The non-symmetry perception enhanced esophageal CT image segmentation method of claim 1, wherein, The is 3, the is 5.

4. The method of claim 1, wherein the asymmetry-awareness based enhancement is applied to the esophageal CT image by using a neural network model. Front The spatial attention bridging module and the channel attention bridging module are sequentially connected on the front one-hop connection path.

5. The method of claim 1, wherein the asymmetry-awareness based esophageal CT image segmentation method is characterized by, The multi-mixed convolution module comprises a parallel visual mamba convolution, a phantom network module, a third addition unit and a second efficient channel attention module; The features input into the multi-mixed convolution module are simultaneously input into the parallel visual Mamba convolution and the phantom network module, a third adding unit is configured to add the features output by the parallel visual Mamba convolution and the features output by the phantom network module, and the features output by the third adding unit are input into the second efficient channel attention module.

6. The method of claim 5, wherein the asymmetry-awareness based enhancement is applied to the esophageal CT image by using a neural network trained by a plurality of training data. The phantom network module comprises: A first parallel branch comprising a first phantom module and a first normalization unit connected in cascade; A second parallel branch comprising a decoupled full connection attention module and a fourth activation function unit connected in cascade; the features input into the phantom network module are simultaneously input into the first phantom module and the decoupled full connection attention module; A fifth multiplication unit configured to multiply the features output by the first normalization unit and the features output by the fourth activation function unit; A second phantom module configured to process the features output by the fifth multiplication unit; A fourth normalization unit configured to normalize the features output by the second phantom module; A first splicing unit configured to splice the features input into the phantom network module and the features output by the fourth normalization unit.

7. The method of claim 6, wherein the asymmetry-awareness based enhancement is applied to the esophageal CT image by using a neural network trained by a plurality of training data. The decoupled full connection attention module comprises a down-sampling processing unit, a sixth convolution layer, a second normalization unit, a horizontal direction full convolution, a third normalization unit, a vertical direction full convolution and a fifth activation function unit connected in cascade.

8. The method of claim 6, wherein the asymmetry-awareness based esophageal CT image segmentation method is characterized by, The first phantom module and the second phantom module both adopt the following structure, comprising: The seventh convolutional layer performs convolution processing on the features input into the first phantom module or the second phantom module to obtain first partial features. convolution processing to obtain first partial features; A depth separable convolution layer configured to process the first part of features by using the depth separable convolution to obtain the second part of features; A second splicing unit configured to splice the first part of features and the second part of features to obtain the features output by the first phantom module or the second phantom module.

Images