Esophageal cancer lesion segmentation system and method based on frequency enhancement

Abstract

The invention provides an esophageal cancer lesion segmentation system and method based on frequency enhancement, and the system comprises a feature extraction module which is used for carrying out the feature extraction of an esophageal cancer lesion digestive endoscopy image through an encoder, and obtaining three advanced features; the feature decoupling module is used for extracting edge information of the image to obtain a global preliminary prediction model; the multi-scale cross-layer fusion module is used for performing cross-layer fusion by using a convolution with different voidage to obtain fusion features; the context feature transformation module is used for fusing the global preliminary prediction model, the edge information and the fused features, and inputting the fused features into a decoder to obtain a global prediction model; and the depth supervision module is used for calculating training loss by utilizing lesion segmentation mask supervision and lesion edge supervision and reducing segmentation errors of the global prediction model. According to the method, the edge of the esophageal lesion in the digestive endoscopy image can be accurately extracted, the segmentation accuracy of the variable-scale lesion can be improved, and effective diagnosis of the early esophageal cancer is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a system and method for segmenting esophageal cancer lesions based on frequency enhancement. Background Art

[0002] Esophageal cancer is a tumor with a high incidence rate. The early symptoms are not obvious. Once discovered, it is often in the late stage. The diagnosis of esophageal cancer mainly relies on digestive endoscopy, but the endoscopic manifestations of early esophageal cancer are atypical and easy to be missed. Therefore, developing a model for automatic segmentation of esophageal cancer lesions is very important for the diagnosis of early esophageal cancer. With the rapid development of artificial intelligence, some segmentation models based on convolutional neural networks have been applied to esophageal digestive endoscopy image recognition.

[0003] However, these segmentation models have the following main defects: (1) Most of them are classic segmentation models of medical images directly transferred to the task of esophageal cancer lesions. However, due to the irregular shape, variable scale, random distribution, and similar color between early lesions and the environment, these classic segmentation models are not effective in directly migrating to esophageal cancer lesions. (2) The existing convolutional neural network-based segmentation model of esophageal cancer lesions cannot capture global and long-range semantic information well. (3) The edge of esophageal cancer lesions is blurred, and the previous segmentation model cannot extract edge information well. Therefore, in view of the above technical shortcomings, the present invention proposes a frequency-enhanced esophageal cancer lesion segmentation system and method. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide an esophageal cancer lesion segmentation system and method based on frequency enhancement, which can not only accurately extract the edges of esophageal lesions, but also improve the segmentation accuracy of lesions of varying scales, thereby achieving effective diagnosis of early esophageal cancer.

[0005] To achieve the above object, the present invention adopts an esophageal cancer lesion segmentation system based on frequency enhancement, comprising:

[0006] A feature extraction module is used to extract features from digestive endoscopic images of esophageal cancer lesions using an encoder to obtain three high-level features;

[0007] A feature decoupling module, used to decouple the high- and low-frequency signals of the three advanced features and perform cross-layer fusion to extract edge information of esophageal lesions and build a global preliminary prediction model. The feature decoupling module includes a signal decoupling unit, a high-frequency signal fusion unit, an edge extraction unit, a low-frequency signal fusion unit, and a preliminary prediction unit;

[0008] A multi-scale cross-layer fusion module, used for performing cross-layer fusion using a convolution with different dilation rates to obtain fusion features; the multi-scale cross-layer fusion module includes a smoothing feature extraction unit, a scale aggregation feature extraction unit and a scale aggregation feature fusion unit;

[0009] A context feature transformation module, used to fuse the esophageal lesion edge information, the global preliminary prediction map and the fusion feature, and input them into a decoder to obtain a global prediction model; the context feature transformation module includes an original feature enhancement unit, an original feature retention unit and a prediction model generation unit;

[0010] A deep supervision module is used to train the global prediction model using a loss function to reduce the segmentation error of the global prediction model; the deep supervision module includes lesion segmentation mask supervision and lesion edge supervision.

[0011] Optionally, the signal decoupling unit is used to decouple each of the high-level features into a high-frequency signal and a low-frequency signal by using octave convolution Fourier transform;

[0012] The high-frequency signal fusion unit is used to fuse the high-frequency signals of the three high-level features to obtain a high-frequency fusion feature;

[0013] The edge extraction unit is used to smoothly output the edge information of the high-frequency fusion feature using two 3×3 convolutions and one 1×1 convolution;

[0014] The low-frequency signal fusion unit is used to fuse the low-frequency signals of the three high-level features to obtain a low-frequency fusion feature;

[0015] The preliminary prediction unit is used to fuse the high-frequency fusion features and the low-frequency fusion features to obtain a complete feature representation, input the complete feature representation into two 3×3 convolutions to obtain global context information, and then input the global context information into a 1×1 convolution to obtain a global preliminary prediction model.

[0016] Optionally, the smooth feature extraction unit is used to input two adjacent high-level features into two 3×3 convolutions in sequence to obtain a four-fold increase in the number of smooth feature channels, and divide the number of smooth feature channels into four groups;

[0017] The scale aggregation feature extraction unit is used to fuse the corresponding groups in two adjacent high-level features based on the four groups of smooth feature channel numbers, and use a 3×3 convolution including four different void rates to capture scale-induced properties to obtain scale aggregation features with different void rates;

[0018] The scale aggregation feature fusion unit is used to fuse the scale aggregation features and then use a 3×3 convolution smoothing to obtain the fused feature.

[0019] Optionally, the original feature enhancement unit is used to fuse the global context information and the fused feature, and obtain a regularized feature after subjecting the fused feature to a 3×3 convolution, global average pooling and nonlinear activation, so as to obtain an enhanced representation of the original feature;

[0020] The original feature retention unit is used to introduce residual connections to obtain the original feature information;

[0021] The prediction model generation unit is used to combine the enhanced representation of the original feature with the original feature information and sequentially pass a 3×3 convolution and a 1×1 convolution to obtain the global prediction model.

[0022] Optionally, the lesion segmentation mask supervision includes weighted IoU loss and binary cross entropy loss, and the lesion edge supervision includes weighted DICE loss.

[0023] The present invention also provides a frequency enhancement-based esophageal cancer lesion segmentation method, comprising the following steps:

[0024] The encoder is used to extract features from digestive endoscopic images of esophageal cancer lesions and obtain three high-level features;

[0025] Based on the advanced features, edge information of the digestive endoscopic image of esophageal cancer lesions is extracted to obtain a global preliminary prediction model;

[0026] Use a convolution with different dilation rates to perform cross-layer fusion to obtain fusion features;

[0027] The feature expression is enhanced by using the attention between global features and local features, and edge details are introduced as guidance to obtain a global prediction model;

[0028] The global prediction model is trained by using lesion segmentation mask supervision and lesion edge supervision to reduce the segmentation error of the global prediction model.

[0029] The present invention discloses the following technical effects by providing a system and method for segmenting esophageal cancer lesions based on frequency enhancement:

[0030] 1. The present invention proposes a frequency-enhanced feature decoupling module, which performs Fourier transform on the three high-level features of the encoder, decouples high- and low-frequency signals and performs cross-layer fusion, thereby achieving accurate extraction of the edge of esophageal lesions and preliminary prediction of the global prediction map.

[0031] 2. The present invention proposes a multi-scale cross-layer fusion module, which performs cross-layer fusion through convolutions with different voiding rates, effectively improving the segmentation accuracy of lesions of varying scales.

[0032] 3. The present invention proposes a contextual feature transformation module, which utilizes the attention between the global prediction map and local features to enhance feature expression, and introduces the esophageal edge as a guide, thereby improving the system's performance in esophageal lesion segmentation.

[0033] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A schematic diagram of a system architecture provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the structure of a feature decoupling module and a context feature transformation module provided in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the structure of a multi-scale cross-layer fusion module provided in an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of a method flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1-3 As shown, the present invention provides an esophageal cancer lesion segmentation system based on frequency enhancement, comprising:

[0042] 1. Feature extraction module, used to extract features of esophageal cancer lesion images using PVT encoder, and obtain three high-level features F i (i=2, 3, 4).

[0043] 2. A feature decoupling module is used to capture multi-scale contextual information and extract the frequency-enhanced edge information provided by the esophageal cancer lesion image based on the high-level features, obtain a global preliminary prediction model, and identify precise lesion boundaries. The feature decoupling module includes a signal decoupling unit, a high-frequency signal fusion unit, an edge extraction unit, a low-frequency signal fusion unit, and a preliminary prediction unit.

[0044] The signal decoupling unit is used to decouple each of the high-level features into a high-frequency signal using an octave convolution Fourier transform. and low frequency signals

[0045] The high-frequency signal fusion unit is used to fuse the high-frequency signals of the three advanced features to obtain a high-frequency fusion feature F H , the formula is:

[0046]

[0047] Among them, F H Represents the high-frequency fusion features of three high-level features, and Respectively represent concatenation and upsampling, The high-frequency signal representing the second high-level feature, The high-frequency signal representing the third high-level feature, High frequency signal representing the fourth high-level feature.

[0048] The edge extraction unit is used to extract the edge information S of the high-frequency fusion feature by using two 3×3 convolutions and one 1×1 convolution. e , the formula is:

[0049]

[0050] Among them, Conv 1×1 represents a convolution, It means that after a 3×3 convolution, it is normalized and nonlinearly activated. Conv 3×3 Indicates a 3×3 convolution without nonlinear activation.

[0051] The low-frequency signal fusion unit is used to fuse the low-frequency signals of the three high-level features to obtain a low-frequency fusion feature F L , the formula is:

[0052]

[0053] Among them, F L Represents the low-frequency fusion features of three high-level features, The low-frequency signal representing the second high-level feature, The low-frequency signal representing the third high-level feature, Low-frequency signal representing the fourth high-level feature.

[0054] Considering that the context information in the low-frequency signal and the boundary attributes in the high-frequency signal are both important for lesion area positioning, the preliminary prediction unit is used to fuse the high-frequency fusion features and the low-frequency fusion features to obtain a complete feature representation, and the complete feature representation is input into two 3×3 convolutions to obtain the global context information F g , and then the global context information F g Input to 1×1 convolution for smoothing to obtain the global preliminary prediction model S g , the formula is:

[0055]

[0056] 3. A multi-scale cross-layer fusion module is used to use a convolution with different void rates to perform cross-layer fusion to obtain fusion features and improve the representation ability of features with different resolutions; the multi-scale cross-layer fusion module includes a smoothing feature extraction unit, a scale aggregation feature extraction unit and a scale aggregation feature fusion unit.

[0057] The smooth feature extraction unit is used to extract the two adjacent high-level features F i and F i+1 The channels are input into two 3×3 convolutions in sequence to obtain a four-fold increase in the number of smooth feature channels. The smooth feature channels are divided into four groups, which are expressed as:

[0058] The scale aggregation feature extraction unit is used to fuse the corresponding groups in two adjacent high-level features based on the four groups of smooth feature channels, and use a 3×3 convolution including four different void rates r={1, 3, 5, 7} to capture the scale-induced properties, so as to obtain scale aggregation features with different void rates. The formula is:

[0059]

[0060] in, represents a convolution with a dilation rate of , which represents concatenation. They respectively represent the features of the corresponding positions of two adjacent encoders.

[0061] The scale aggregation feature fusion unit is used to fuse the scale aggregation features and then use a 3×3 convolution smoothing to obtain the fused feature The formula is:

[0062]

[0063] 4. A context feature transformation module, which is used to fuse the global preliminary prediction model, the edge information and the fused features, and input them into the decoder to obtain a global prediction model to improve the accuracy of esophageal lesion boundary segmentation; the context feature transformation module includes an original feature enhancement unit, an original feature retention unit and a prediction model generation unit.

[0064] The original feature enhancement unit is used to fuse the global context information F g and the fusion feature (with Take the fused features as an example). After a 3×3 convolution, global average pooling and nonlinear activation, a regularized feature W is obtained, which is an enhanced representation of the original feature. The formula of the regularized feature W is:

[0065]

[0066] in, represents the global average pooling, σ represents the nonlinear activation function Sigmoid, F mer It represents a fusion feature of global and local.

[0067] The original feature retention unit is used to introduce residual connections to obtain the original feature information. In order to explore the correlation between global and local modalities, the regularized feature W can be regarded as an attention map to strengthen the feature representation. In addition, the residual connection is introduced while combining the features of the previous decoder, which can make the network more effective and retain the original feature information. Therefore, the formula for introducing regularized features and residual connections is:

[0068]

[0069] Among them, Conv 1×1 represents a convolution, It means element-wise multiplication. Indicates that the elements are added together.

[0070] Then, the edge information S e Boundary enhancement information is provided to weigh features in the context transformation module.

[0071] The model generation unit is used to combine the enhanced representation of the original feature with the original feature information and sequentially pass a 3×3 convolution and a 1×1 convolution to obtain the global prediction model. and S e Multiply and then Add together and after a 3×3 convolution we get The formula is:

[0072]

[0073] Finally, A 1×1 convolution is performed to obtain the global prediction model S3. Similarly, the context feature transformation module can be used to obtain the global prediction models S2 and S1.

[0074] 5. A deep supervision module, used to train the global prediction model using a loss function to reduce the segmentation error of the global prediction model; the deep supervision module includes lesion segmentation mask supervision and lesion edge supervision.

[0075] The lesion segmentation mask G t Supervision includes weighted IoU loss and binary cross entropy loss Lesion segmentation mask G t The supervised loss function is defined as

[0076] The lesion edge monitoring G e Including weighted DICE loss Lesion edge monitoring G e The loss function is defined as

[0077] The overall loss function is defined as:

[0078]

[0079] like Figure 4 As shown, the present invention also provides an esophageal cancer lesion segmentation method based on frequency enhancement, comprising the following steps:

[0080] The PVT encoder is used to extract features from esophageal cancer lesion images and obtain three high-level features;

[0081] Based on the high-level features, edge information of the esophageal cancer lesion image is extracted to obtain a global preliminary prediction model;

[0082] Use a convolution with different dilation rates to perform cross-layer fusion to obtain fusion features;

[0083] fusing the global preliminary prediction model, the edge information and the fusion feature, and inputting them into a decoder to obtain a global prediction model;

[0084] The global prediction model is trained by using lesion segmentation mask supervision and lesion edge supervision to reduce the segmentation error of the global prediction model.

[0085] Therefore, the present invention provides an esophageal cancer lesion segmentation system and method based on frequency enhancement, which can not only accurately extract the edges of esophageal lesions, but also improve the segmentation accuracy of lesions with varying scales, thereby achieving effective diagnosis of early esophageal cancer.

[0086] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0087] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A frequency-enhanced esophageal cancer lesion segmentation system, characterized in that: include: A feature extraction module is used to extract features from digestive endoscopic images of esophageal cancer lesions using an encoder to obtain three high-level features; A feature decoupling module, used to decouple the high- and low-frequency signals of the three advanced features and perform cross-layer fusion to extract edge information of esophageal lesions and build a global preliminary prediction model. The feature decoupling module includes a signal decoupling unit, a high-frequency signal fusion unit, an edge extraction unit, a low-frequency signal fusion unit, and a preliminary prediction unit; A multi-scale cross-layer fusion module, used for performing cross-layer fusion using a convolution with different dilation rates to obtain fusion features; the multi-scale cross-layer fusion module includes a smoothing feature extraction unit, a scale aggregation feature extraction unit and a scale aggregation feature fusion unit; A context feature transformation module, used to fuse the esophageal lesion edge information, the global preliminary prediction map and the fusion feature, and input them into a decoder to obtain a global prediction model; the context feature transformation module includes an original feature enhancement unit, an original feature retention unit and a prediction model generation unit; A deep supervision module is used to train the global prediction model using a loss function to reduce the segmentation error of the global prediction model; the deep supervision module includes lesion segmentation mask supervision and lesion edge supervision.

2. The frequency-enhanced esophageal cancer lesion segmentation system according to claim 1, characterized in that: The signal decoupling unit is used to decouple each of the high-level features into a high-frequency signal and a low-frequency signal by using octave convolution Fourier transform; The high-frequency signal fusion unit is used to fuse the high-frequency signals of the three high-level features to obtain a high-frequency fusion feature; The edge extraction unit is used to smooth the edge information of the output image using two 3×3 convolutions and one 1×1 convolution; The low-frequency signal fusion unit is used to fuse the low-frequency signals of the three high-level features to obtain a low-frequency fusion feature; The preliminary prediction unit is used to fuse the high-frequency fusion features and the low-frequency fusion features to obtain a complete feature representation, input the complete feature representation into two 3×3 convolutions to obtain global context information, and then input the global context information into a 1×1 convolution to obtain a global preliminary prediction model.

3. The frequency-enhanced esophageal cancer lesion segmentation system according to claim 2, characterized in that: The smooth feature extraction unit is used to input two adjacent high-level features into two 3×3 convolutions in sequence to obtain a four-fold increase in the number of smooth feature channels, and divide the number of smooth feature channels into four groups; The scale aggregation feature extraction unit is used to fuse the corresponding groups in two adjacent high-level features based on the four groups of smooth feature channel numbers, and use a 3×3 convolution including four different void rates to capture scale-induced properties to obtain scale aggregation features with different void rates; The scale aggregation feature fusion unit is used to fuse the scale aggregation features and then use a 3×3 convolution smoothing to obtain the fused feature.

4. The frequency-enhanced esophageal cancer lesion segmentation system according to claim 3, characterized in that: The original feature enhancement unit is used to fuse the global context information and the fused feature, and obtain a regularized feature after the fused feature undergoes a 3×3 convolution, global average pooling and nonlinear activation to obtain an enhanced representation of the original feature; The original feature retention unit is used to introduce residual connections to obtain the original feature information; The prediction model generation unit is used to combine the enhanced representation of the original feature with the original feature information and sequentially pass a 3×3 convolution and a 1×1 convolution to obtain the global prediction model.

5. The frequency-enhanced esophageal cancer lesion segmentation system according to claim 4, characterized in that: The lesion segmentation mask supervision includes weighted IoU loss and binary cross entropy loss, and the lesion edge supervision includes weighted DICE loss.

6. A frequency-enhanced esophageal cancer lesion segmentation method, characterized in that: The following steps are involved: The encoder is used to extract features from digestive endoscopic images of esophageal cancer lesions and obtain three high-level features; Based on the advanced features, edge information of the digestive endoscopic image of esophageal cancer lesions is extracted to obtain a global preliminary prediction model; Use a convolution with different dilation rates to perform cross-layer fusion to obtain fusion features; The feature expression is enhanced by using the attention between global features and local features, and edge details are introduced as guidance to obtain a global prediction model; The global prediction model is trained by using lesion segmentation mask supervision and lesion edge supervision to reduce the segmentation error of the global prediction model.

Images