Endoscope polyp image segmentation method based on attention mechanism and style mixing technology

Abstract

The application discloses an endoscope polyp image segmentation method based on an attention mechanism and a style mixing technology, which comprises an endoscope polyp image segmentation method based on an attention mechanism and a style mixing technology, an encoder-decoder architecture, construction of a polyp image segmentation model basic architecture, use of MSCAN as an encoder, fusion and extraction of multi-scale features, application of a MixStyle technology between stages of the encoder, and improvement of the domain generalization performance of the model. In the decoder, UAM is used to aggregate multi-scale features generated by the encoder, and then the aggregated features are sequentially detected for more important context information from a channel and a spatial representation; OAM is used to track target and complementary edge information, to build a connection between the encoder and decoder features, to reduce the distribution difference between the two features, and to obtain a more accurate segmentation graph. In the training process, the application uses an API loss to guide the training of the model in an adaptive manner, and improves the performance of the polyp image segmentation model as a whole.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to an endoscopic polyp image segmentation method based on attention mechanism and style mixing technology. Background Technology

[0002] Gastrointestinal cancers are on the rise, with colorectal cancer ranking among the leading causes of new cases and deaths worldwide. Colorectal cancer is a common digestive tract cancer, with most cases caused by colorectal adenomatous polyps. Studies and experiments have shown that the survival rate of colorectal cancer patients is closely related to the stage of the polyps; the later the polyps are detected, the lower the survival rate. The earlier colorectal cancer is detected, diagnosed, and treated, the greater the chance of saving a patient's life. Therefore, early screening and treatment of colorectal cancer are of great significance.

[0003] Currently, colonoscopy is an effective method for screening colorectal polyps and is considered the "gold standard" for intestinal disease screening. Through endoscopy, doctors can directly observe the size and shape of polyps, as well as the lesions in intestinal tissue, enabling clinical diagnosis and treatment of intestinal diseases. However, on the one hand, traditional endoscopy relies entirely on the doctor's manual observation and judgment, making it susceptible to subjective interference during diagnosis. Furthermore, prolonged work can lead to visual fatigue for doctors, resulting in missed or false diagnoses. On the other hand, because the size and shape of polyps and lesions vary from patient to patient, there are often tiny polyps with very small diameters that are difficult for doctors to detect during clinical examination. Therefore, utilizing computer-aided technology to assist doctors in diagnosis and reduce missed or false diagnoses during endoscopy is particularly important.

[0004] In recent years, deep learning has made significant strides in the field of polyp image segmentation. However, deep learning models often rely on the assumption that training and test data are identically distributed. When this assumption is violated, the model's performance drops dramatically. Research in this area is known as domain generalization (DG). For polyp images, each patient's intestinal tissue differs, with varying degrees of lesions, polyp types, and morphologies, exhibiting diverse styles. Training data cannot encompass the statistical distribution of polyp images from all patients, and in practical applications, models often face polyp images from completely unfamiliar patients. Therefore, developing a technique to optimize polyp image segmentation algorithms from a domain generalization perspective, thereby improving polyp image segmentation performance, is particularly important for clinical applications. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology. To achieve the above objective, an endoscopic polyp image segmentation method based on attention mechanism and style mixing technology is adopted to solve the problems mentioned in the background technology.

[0006] An endoscopic polyp image segmentation method based on attention mechanism and style fusion technique includes the following steps:

[0007] Step S1: Obtain the endoscopic examination video, and annotate the polyp area in the video at the pixel level to obtain the true segmentation label;

[0008] Step S2: Based on the encoder-decoder architecture, construct the basic architecture of the polyp image segmentation model, where the encoder is used to extract multi-scale features of the input image, and the decoder is used to process the features obtained by the encoder to obtain the segmentation prediction map.

[0009] Step S3: Construct the encoder MSCAN using four stages with progressively decreasing resolution, generating four feature maps of different scales. Each stage contains multiple building blocks, and the basic unit of the building blocks is the MSCA module.

[0010] Step S4: After the first three stages of the encoder, the MixStyle technique is added for implicit data augmentation to perturb the style information of the source domain training instances to regularize the training of the model.

[0011] Step S5: In the decoder, the UAM module is used to aggregate the multi-scale features generated by the encoder, and then more contextual information is detected from the channel and spatial representations of the aggregated features in turn.

[0012] Step S6: In the decoder, the OAM module is used to track the target and complementary edge information, build the connection between the encoder and decoder features, reduce the distribution difference between the two features, and obtain an accurate segmentation map.

[0013] Step S7: In the OAM module, generate multiple segmentation prediction maps of different sizes and interpolate them to the same size as the real labels. Calculate the loss value for each map during training to introduce deep supervision.

[0014] Step S8: During model training, combine three loss functions—BCE loss, IoU loss, and L1 loss—and define pixel intensity ω to obtain API loss, and use an adaptive approach to guide model training.

[0015] Step S9: Combine the networks from steps S2 to S7 and train them using the data from step S1. Calculate the training loss according to the method in step S8. After training, obtain the final model and export the model for deployment and inference.

[0016] As a further technical solution of the present invention, the specific steps in step S1 include:

[0017] Raw video data without personal information was collected from real hospital endoscopic diagnostic scenarios, containing only the lesion name and time period, and was annotated under the guidance of specialist doctors;

[0018] The preprocessing mainly includes splitting video frames, removing endoscope manufacturer information, and unifying the size. Labelme is used to manually label polyp lesions as positive samples, while automated tools generate negative samples without polyps.

[0019] Finally, the datasets were divided into training, validation, and test datasets in an 8:1:1 ratio.

[0020] As a further technical solution of the present invention: the specific steps in step S2 of constructing the basic architecture of the polyp image segmentation model based on the encoder-decoder architecture include:

[0021] The encoder is MSCAN encoder, used to extract multi-scale features from the input image;

[0022] The encoder MSCAN outputs four layers of feature maps, denoted as E1, E2, E3 and E4 respectively, and the latter three layers of feature maps will be input into the decoder to perform the segmentation task.

[0023] The decoder includes a UAM module and an OAM module. The UAM module aggregates the features of the last three layers of feature maps output by the encoder, and the OAM module interacts with the aggregated feature map and the original feature map to obtain the final segmentation prediction map.

[0024] As a further technical solution of the present invention, the specific steps in step S3 include:

[0025] The MSCA module comprises three parts: a deep convolution for aggregating local information, a multi-branch deep strip convolution for capturing multi-scale context, and a 1×1 convolution for modeling the relationship between different channels. The output of the 1×1 convolution is used as attention weights to weight the input.

[0026] The expression for the MSCA module is:

[0027]

[0028] Where E represents the input feature, This represents a 1×1 convolution, where α and X represent the attention map and output, respectively. DWConv represents element-wise matrix multiplication, DWConv represents depthwise convolution, and Scale represents scale. i,i∈{0,1,2,3} represents the i-th branch in MSCA, and Scale0 is the direct branch.

[0029] As a further technical solution of the present invention: in each branch, the MSCA module uses two lightweight depth band convolutions to approximate the standard depth convolution with a large kernel;

[0030] The MSCAN encoder comes in four different sizes: MSCAN-T, MSCAN-S, MSCAN-B, and MSCAN-L. The difference between the four sizes lies in the number of blocks contained in each stage and the number of output channels.

[0031] As a further technical solution of the present invention, the specific steps in step S4 include:

[0032] MixStyle, a style blending technique, inserts plug-and-play modules between the layers of a CNN to blend the feature statistics of two instances according to certain weights, in order to simulate new styles.

[0033] As a further technical solution of the present invention: given an input batch x, a reference batch is first generated from x based on the style mixing technique MixStyle. Because x is obtained by randomly sampling from all training data, the data may randomly come from multiple domains, therefore It can be done get;

[0034] generate Then, MixStyle calculates the mixture feature statistic according to the following formula:

[0035]

[0036] in, It is sampled from the Beta distribution, i.e., λ ~ Beta(α,α), where α∈(0,∞) is a hyperparameter;

[0037] Finally, applying the mixed feature statistics to x, we get:

[0038]

[0039] As a further technical solution of the present invention, the specific steps in steps S5 and S6 include:

[0040] The last three layer outputs E2, E3, and E4 of the encoder are aggregated to channels 32, 64, and 128 respectively, and then the following operations are performed to obtain the aggregated representation:

[0041]

[0042] E″3=f(cat[E′3,f(Up(E4))])

[0043] E″2=f(Up(E″3))

[0044] X = f(cat[E′2,E″2])

[0045] Where Up(·) represents upsampling operation, f(·) represents convolution operation, and cat(·) represents channel-wise feature concatenation;

[0046] Features after aggregation The size is the same as E2; after aggregation, the relatively important channel context information is first distinguished, and then the spatial information is enhanced based on the complementary confidence scores obtained from the channel context.

[0047] By utilizing the softmax and sigmoid functions and employing self-attention, the relationships between channels are obtained, and features from different channels are aggregated to obtain attention scores.

[0048]

[0049] Next, to refine the aggregate representation X, channel attention is used, applying confidence channel weights and adjusting according to α. c The distribution and confidence ratio γ retain the confidence channel:

[0050]

[0051] Among them, F -1 (γ) represents α c The γ quantiles are then calculated; subsequently, the refined input is computed using spatial attention. Used to distinguish salient targets and generate the first decoder representation. The formula is:

[0052]

[0053] in, The input features are projected onto a 1×1 convolution. Finally, upsample D0 to DS0 to obtain deep supervision.

[0054] As a further technical solution of the present invention, the specific steps in step S7 include:

[0055] By reducing the distributional differences between the encoder and decoder feature representations with fewer parameters, a more accurate segmentation map can be obtained, where D0 is the output of the UAM module;

[0056] OAM is represented from each decoder. The tracking focuses on the target and complementary edge information, where the target weight α O The calculation method is as follows:

[0057] α O =σ(D i )

[0058] Where σ(·) represents the sigmoid function; however, α O It is impossible to always detect a clear edge region of the entire object; therefore, a complementary edge weight α is generated. E To cover undetected areas;

[0059] For each pixel x in D ij The detected region is inverted, and the background noise corresponding to the noise reduction ratio d is eliminated to perform missed region detection. The formula is as follows:

[0060]

[0061] Combine the encoder output features E1 and E2 with the decoder features D1 and D2, and then... i+1 Perform upsampling operation to generate DS i+1 ;

[0062]

[0063] Combining the above parts, we obtain the final completed model.

[0064] As a further technical solution of the present invention, the specific steps in step S8 include:

[0065] The loss function used during training is API loss, which combines three loss functions: BCE loss, IoU loss, and L1 loss, and uses an adaptive approach to guide the training of the model.

[0066] The API loss defines the pixel intensity ω and applies it to each pixel, giving greater weight to pixels closer to the edges of the detected object. The formula is:

[0067]

[0068] Where λ is the penalty term and K is the kernel size;

[0069] Using ω to improve the BCE loss, we have adaptive BCE(aBCE)loss; where y c and These are the actual value and the predicted value, respectively; by using ω, aBCE loss, the network can pay more attention to the local structure at the edge;

[0070]

[0071] Using ω to improve IoU loss, we get adaptive IoU(aIoU)loss; aIoU loss can focus more on features related to dense regions and optimize the global structure.

[0072]

[0073] ω is used to improve L1 loss. L1 loss itself allows the network to learn robustly to noisy labels, while the improved L1 loss can better distinguish relatively important pixels and eliminate noisy pixels.

[0074]

[0075] By optimizing the local and global structures and combining the improved losses described above, we can obtain the API loss, as shown in the formula:

[0076]

[0077] Finally, the three deep supervisions (DS0, DS1, and DS2) output by the decoder, and the set of the three supervisions DS, are respectively processed. e Feed in the ground truth (GT) Calculate the loss values ​​and sum them to obtain the final loss function of the model, as shown in the formula:

[0078]

[0079] Compared with the prior art, the present invention has the following technical advantages:

[0080] Employing the aforementioned technical solution, an endoscopic polyp image segmentation method based on attention mechanisms and style mixing techniques is used. An encoder-decoder architecture is established to construct the basic framework of the polyp image segmentation model. MSCAN is used as the encoder to fuse and extract multi-scale features, and MixStyle technology is applied between encoder stages to improve the model's domain generalization performance. In the decoder, UAM is used to aggregate the multi-scale features generated by the encoder, and then more important contextual information is detected from the channel and spatial representations of the aggregated features. OAM is used to track the target and complementary edge information, building a connection between encoder and decoder features, reducing the distribution differences between the two features, and obtaining a more accurate segmentation map. During training, API loss is used to adaptively guide the model training, improving the overall performance of the polyp image segmentation model. Attached Figure Description

[0081] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings:

[0082] Figure 1 This is a schematic diagram illustrating the steps of the endoscopic polyp image segmentation method according to an embodiment of this application;

[0083] Figure 2 This is a visualization of the segmentation results of an embodiment disclosed in this application. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Please refer to Figure 1 and Figure 2 In this embodiment of the invention, an endoscopic polyp image segmentation method based on attention mechanism and style mixing technology includes the following steps:

[0086] Step S1: Obtain endoscopic examination videos from the hospital, remove redundant information, and perform pixel-level annotation on polyp areas to obtain accurate segmentation labels. The original video data is collected from real hospital endoscopic diagnostic scenarios and, to address privacy and ethical concerns, excludes all patient personal information, including only the lesion name and the time period in which it appears. A report indicating the removal of personal information is included with the data. Based on this, a specialist guides the data annotation process. During preprocessing and annotation, the video is split into frames, and information from the endoscope manufacturer, such as timestamps, is removed. All frames are standardized to 1160×1080 pixels. Using LabelMe, positive samples are generated by outlining the edges of polyp lesions with a white foreground (polyp) and a black background (intestine). Frames without polyps are automatically labeled as negative samples (completely black). Positive and negative samples share the same name and are divided into training, validation, and test datasets in an 8:1:1 ratio.

[0087] Step S2: Based on the encoder-decoder architecture, construct the basic architecture of the polyp image segmentation model. The encoder extracts multi-scale features from the input image, and the decoder processes these features to obtain a segmentation prediction map. The encoder is MSCAN, used to extract multi-scale features from the input image. MSCAN outputs four feature maps, denoted as E1, E2, E3, and E4. The latter three feature maps are input to the decoder to perform the segmentation task. The decoder includes UAM and OAM. UAM aggregates the features from the latter three feature maps output by the encoder, and OAM interacts with the aggregated feature map and the original feature map to obtain the final segmentation prediction map.

[0088] Step S3: Construct the encoder MSCAN using four stages with progressively decreasing resolution, generating four feature maps at different scales. Each stage contains multiple building blocks, with the basic unit of each building block being the MSCA module. The core idea of ​​MSCA is to replace large-kernel convolutions with depthwise strip convolutions and construct spatial attention, while extending the structure to multiple scales for better fusion of multi-scale features. MSCA consists of three parts: a depth-wise convolution (DWConv) for aggregating local information, a multi-branch depthwise strip convolution for capturing multi-scale context, and a 1×1 convolution (1×1Conv) for modeling the connections between different channels. The output of the 1×1 convolution is directly used as attention weights to weight the input. The mathematical expression of MSCA is as follows:

[0089]

[0090] Where E represents the input feature. This represents a 1×1 convolution, where α and X represent the attention map and output, respectively. DWConv represents element-wise matrix multiplication, DWConv represents depthwise convolution, and Scale represents scale. i Let i ∈ {0,1,2,3} represent the i-th branch in MSCA, where Scale0 is the directly connected branch. In each branch, MSCA uses two depthwise striped convolutions to approximate a large-kernel standard depthwise convolution, with kernel sizes of 7, 11, and 21 for each branch. Depthwise striped convolutions are chosen because they are lightweight; for example, to simulate a standard 2D convolution with a kernel size of 11×11, only a pair of 7×1 and 1×7 convolutions are needed. Stacking a series of blocks forms the convolutional encoder, MSCAN. MSCAN employs a general hierarchical structure, containing four stages with decreasing spatial resolution, where the resolutions are... and Here, H and W represent the height and width of the input image, respectively. Each stage contains a downsampling module and a series of blocks. The downsampling module includes a 3×3 convolution with a stride of 2 and a batch normalization (BN) layer. MSCAN has four different sizes: MSCAN-T, MSCAN-S, MSCAN-B, and MSCAN-L, differing in the number of blocks and output channels in each stage. We chose MSCAN-T as the backbone, with each stage containing 3, 3, 5, and 2 blocks, and output channels of 32, 64, 160, and 256, respectively.

[0091] Step S4: After the first three stages of the encoder, a style mixing (MixStyle) technique is added for implicit data augmentation. This aims to perturb the style information of the source domain training instances to regularize the model's training and improve its domain generalization performance. MixStyle can be easily integrated into mini-batch training. Given an input batch x, MixStyle first generates a reference batch from x. Because x is obtained by randomly sampling from all training data, the data may randomly come from multiple domains, therefore It can be done Obtain. Generate Then, MixStyle calculates the mixture feature statistics as follows:

[0092]

[0093] in It is sampled from the Beta distribution, i.e., λ ~ Beta(α,α), where α∈(0,∞) is a hyperparameter set to 0.1. Finally, the mixed feature statistics are applied to x to obtain:

[0094]

[0095] During training, we use a 50% probability to decide whether to apply MixStyle during forward propagation. During testing, MixStyle is not applied.

[0096] Step S5: In the decoder, the multi-scale features generated by the encoder are aggregated using UAM, and then more important contextual information is detected from the channel and spatial representations of the aggregated features sequentially. UAM first aggregates the outputs of the last three layers of the encoder, E2, E3, and E4, to 32, 64, and 128 channels respectively, and then performs the following operations to obtain the aggregated representation:

[0097]

[0098] E″3=f(cat[E′3,f(Up(E4))])#(8)

[0099] E″2=f(Up(E″3))#(9)

[0100] X=f(cat[E′2,E″2])#(10)

[0101] Where Up(·) represents the upsampling operation, f(·) represents the convolution operation, and cat(·) represents the channel-wise feature concatenation. The aggregated features... The dimensions are the same as E2. After aggregation, relatively important channel context information is first distinguished, and then spatial information is enhanced based on complementary confidence scores obtained from the channel context. As shown in Equation 11, This represents the characteristics after channel pooling. This represents a 1×1 convolution. Using the softmax and sigmoid functions (denoted by σ(·)), self-attention is employed to obtain the relationships between channels and aggregate features from different channels to obtain attention scores.

[0102]

[0103] Next, to refine the aggregate representation X, channel attention is used, applying confidence channel weights and adjusting according to α. c The distribution and confidence ratio γ retain the confidence channel:

[0104]

[0105] Among them, F -1 (γ) represents α c The γ quantile is then calculated using spatial attention to refine the input. Used to distinguish salient targets and generate the first decoder representation. as follows:

[0106]

[0107] in, The input features are projected onto a 1×1 convolution. Finally, upsampling D0 to DS0 yields deep supervision.

[0108] Step S6: In the decoder, OAM is used to track the target and complementary edge information, constructing a connection between the encoder and decoder features, reducing the distribution difference between the two features, and obtaining a more accurate segmentation map. To improve decoder efficiency, D will remain a single channel. OAM is derived from each decoder representation. The tracking focuses on the target and complementary edge information, where the target weight α O The calculation method is as follows:

[0109] α O =σ(D i )#(15)

[0110] Where σ(·) represents the sigmoid function. However, α O It is impossible to always detect a clear edge region of the entire object; therefore, a complementary edge weight α is generated. E To cover undetected areas. For each pixel x in D ij The detected region is inverted, and the background noise corresponding to the noise reduction ratio d is removed to perform missed region detection, as follows:

[0111]

[0112] Furthermore, the encoder output features E1 and E2 are combined with the decoder features D1 and D2, and D is then processed. i+1 Perform upsampling operation to generate DS i+1 .

[0113]

[0114] Step S7: In the OAM module, generate multiple segmentation prediction maps of different sizes and interpolate them to the same size as the real labels. Calculate the loss value for each map during training to introduce deep supervision.

[0115] Step S8: Train the completed model using the prepared data, combining the three loss functions: BCE loss, IoU loss, and L1 loss, and defining the pixel intensity ω to form the API loss. An adaptive approach is used to guide the model training. The API loss defines the pixel intensity ω and applies it to each pixel, giving greater weight to pixels closer to the edges of the detected object, as follows:

[0116]

[0117] Where λ is the penalty term and K is the kernel size, empirically set to 0.5 and K to K∈{3,15,31}. Applying ω to improve the BCE loss, we have the adaptive BCE(aBCE)loss, as shown in equation (3-16), where y c and These are the true value and the predicted value, respectively. By using ω, the aBCE loss enables the network to pay more attention to the local structure at the edges.

[0118]

[0119] Using ω to improve the IoU loss, we get the adaptive IoU(aIoU)loss, as shown in Equation (3-17). Compared to the original IoU loss, aIoU loss focuses more on features related to dense regions, optimizing the global structure.

[0120]

[0121] ω is used to improve the L1 loss, as shown in Equation (3-18). The L1 loss itself allows the network to learn robustly against noisy labels, while the improved L1 loss can better distinguish relatively important pixels and eliminate noisy pixels.

[0122]

[0123] To comprehensively optimize both local and global structures, the three improved losses mentioned above are combined to obtain the API loss:

[0124]

[0125] Finally, the three deep supervisions (DS0, DS1, and DS2) output by the decoder, and the set of the three supervisions DS, are respectively processed. e Feed in the ground truth (GT) Calculate the loss values ​​and sum them to obtain the final loss function of the model, as follows:

[0126]

[0127] Step S9: After training, the test data can be used for inference, or the model can be exported as an ONNX model for deployment and inference on embedded devices. The specific deployment and inference process is as follows: Transfer the optimal model to the Jetson Xavier NX development board, configure the corresponding Python environment on the development board, and use the Python-based model conversion tool torch2trt to convert the optimal model into an FP16 precision TensorRT model on the development board; use the QT framework based on C++ language to write the deployment and inference code of the TensorRT model on the development board to ensure that the input polyp image or video can obtain polyp segmentation results and display them in the graphical interface after passing through the TensorRT model; obtain the actual endoscopy scene data and input it into the deployed model to obtain the polyp segmentation results, determine the location of the polyp, and assist the doctor in diagnosis.

[0128] In this embodiment, the SUN-SEG dataset was used, employing its officially defined training set and testing under its easy / unseen and hard / unseen test set standards. Superior experimental results were achieved on multiple deep learning metrics for image segmentation. The official project address for this dataset is as follows: https: / / github.com / GewelsJI / VPS. The test results of this invention are shown in Tables 1 and 2.

[0129] Table 1

[0130]

[0131] Table 2

[0132]

[0133] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention. The scope of the invention is defined by the appended claims and their equivalents, all of which should be included within the scope of protection of the invention.

Claims

1. A method for segmenting endoscopic polyp images based on attention mechanisms and style fusion techniques, characterized in that, Includes the following steps: Step S1: Obtain the endoscopic examination video, annotate the polyp area in the video at the pixel level, and obtain the true segmentation label; Step S2: Based on the encoder-decoder architecture, construct the basic architecture of the polyp image segmentation model, where the encoder is used to extract multi-scale features of the input image, and the decoder is used to process the features obtained by the encoder to obtain the segmentation prediction map. Step S3: Construct the encoder MSCAN using four stages with progressively decreasing resolution, generating four feature maps of different scales. Each stage contains multiple building blocks, and the basic unit of the building blocks is the MSCA module. Step S4: After the first three stages of the encoder, the MixStyle style mixing technique is added for implicit data augmentation to perturb the style information of the source domain training instances in order to regularize the training of the model. Step S5: In the decoder, the UAM module is used to aggregate the multi-scale features generated by the encoder, and then the context information is detected from the channel and spatial representations of the aggregated features in turn. Step S6: In the decoder, the OAM module is used to track the target and complementary edge information, build the connection between the encoder and decoder features, reduce the distribution difference between the two features, and obtain an accurate segmentation map. Step S7: In the OAM module, generate multiple segmentation prediction maps of different sizes and interpolate them to the same size as the real labels. Calculate the loss value for each map during training to introduce deep supervision. Step S8: During model training, combine three loss functions: BCE loss, IoU loss, and L1 loss, and define pixel intensity. ω The API loss is obtained, and an adaptive approach is used to guide the training of the model. The specific steps include: The loss function used during training is API loss, which combines three loss functions: BCE loss, IoU loss, and L1 loss, and adopts an adaptive approach to guide the training of the model. API loss defines pixel intensity It is then applied to each pixel, giving greater weight to pixels closer to the edges of the detected object, as shown in the formula: in, It is a penalty item. It refers to the size of the nucleus; Will To improve the BCE loss, we have the adaptive BCE(aBCE) loss; where and These are the actual value and the predicted value; by using aBCE loss enables the network to pay more attention to local structures at the edges; Will To improve IoU loss, there is adaptive IoU (aIoU) loss; aIoU loss can focus more on features related to dense regions and optimize the global structure. Will Used to improve L1 loss. L1 loss itself enables the network to learn robustly to noisy labels, while the improved L1 loss can better distinguish relatively important pixels and eliminate noisy pixels. By optimizing the local and global structures and combining the improved losses described above, we can obtain the API loss, as shown in the formula: Finally, the three deep supervisory layers output by the decoder are respectively... , and And a set of three supervisors Input with the actual value Calculate the loss values ​​and sum them to obtain the final loss function of the model, as shown in the formula: ； Step S9: Combine the networks from steps S2 to S7 and train them using the data from step S1. Calculate the training loss according to the method in step S8. After training, obtain the final model and export the model for deployment and inference.

2. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 1, characterized in that, The specific steps in step S1 include: Raw video data without personal information was collected from real hospital endoscopic diagnostic scenarios, containing only the lesion name and time period, and was annotated under the guidance of specialist doctors; The preprocessing mainly includes splitting video frames, removing endoscope manufacturer information, and unifying the size. Labelme is used to manually label polyp lesions as positive samples, while automated tools generate negative samples without polyps. Finally, the datasets were divided into training, validation, and test datasets in an 8:1:1 ratio.

3. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 1, characterized in that, The specific steps in step S2, which involve constructing the basic architecture of the polyp image segmentation model based on the encoder-decoder architecture, include: The encoder is MSCAN encoder, used to extract multi-scale features from the input image; The encoder MSCAN outputs four layers of feature maps, which are represented as follows: , , and Furthermore, the feature maps of the last three layers will be input into the decoder to perform the segmentation task; The decoder includes a UAM module and an OAM module. The UAM module aggregates the features of the last three layers of feature maps output by the encoder, and the OAM module interacts with the aggregated feature map and the original feature map to obtain the final segmentation prediction map.

4. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 1, characterized in that, The specific steps in step S3 include: The MSCA module comprises three parts: a deep convolution for aggregating local information, a multi-branch deep strip convolution for capturing multi-scale context, and a 1×1 convolution for modeling the relationship between different channels. The output of the 1×1 convolution is used as attention weights to weight the input. The expression for the MSCA module is: in, Indicates input features, Represents a 1×1 convolution. and These represent the attention map and the output, respectively. DWConv represents element-wise matrix multiplication, and DWConv represents depthwise convolution. This represents the i-th branch in the MSCA. It is a direct branch.

5. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 4, characterized in that, In each branch, the MSCA module uses two lightweight depth band convolutions to approximate a standard depth convolution with a large kernel. The encoder MSCAN comes in four different sizes: MSCAN-T, MSCAN-S, MSCAN-B, and MSCAN-L. The difference between the four sizes lies in the number of blocks contained in each stage and the number of output channels.

6. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 1, characterized in that, The specific steps in step S4 include: MixStyle, a style blending technique, inserts plug-and-play modules between the layers of a CNN to blend the feature statistics of two instances according to certain weights, in order to simulate new styles.

7. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 6, characterized in that, Given an input batch Based on the style mixing technology MixStyle, it first started from Generate a reference batch ,because It is obtained by randomly sampling from all training data, and the data may randomly come from multiple domains, so able to pass get; generate Then, MixStyle calculates the mixture feature statistic according to the following formula: in, It was sampled from the Beta distribution, i.e. , It is a hyperparameter; Finally, the mixed feature statistics are applied to ,get:

8. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 1, characterized in that, The specific steps in steps S5 and S6 include: Output the last three layers of the encoder , and Aggregate to 32, 64, and 128 channels respectively, and then perform the following operations to obtain the aggregated representation: in, Indicates an upsampling operation. This represents the convolution operation. Indicates channel-by-channel feature connection; Features after aggregation Size and The same; after aggregation, the relatively important channel context information is first distinguished, and then the spatial information is enhanced based on the complementary confidence scores obtained from the channel context. By utilizing the softmax and sigmoid functions and employing self-attention, the relationships between channels are obtained, and features from different channels are aggregated to obtain attention scores. : Next, in order to refine the aggregation representation Using channel attention, apply confidence channel weights, and according to Distribution and confidence ratio Preserving Confidence Channels: in, express of Quantiles; then, spatial attention is used to refine the input. This is used to distinguish salient targets and generate the first decoder representation. The formula is: in, The input features are projected onto a 1×1 convolution. Up; finally, Upsampling This allows for in-depth oversight.

9. The endoscopic polyp image segmentation method based on attention mechanism and style mixing technology according to claim 1, characterized in that, The specific steps in step S7 include: By reducing the distributional differences between the encoder and decoder feature representations with fewer parameters, a more accurate segmentation map can be obtained. This is the output of the UAM module; OAM is represented from each decoder. Tracking targets and complementary edge information, where target weights The calculation method is as follows: in, This represents the sigmoid function; however, It's impossible to always detect a clear edge region of the entire object; therefore, a complementary edge weight is generated. To cover undetected areas; for Each pixel in The detected region is inverted and eliminated, resulting in a lower noise reduction ratio. The corresponding background noise is used to detect missed areas, and the formula is: Encoder output features , With decoder features , Combined together, and for Generate by performing upsampling operation ; Combining the above parts, we obtain the final completed model.

Images