Cervical image processing method based on double attention and multi-scale fusion
Through deep learning methods based on dual attention and multi-scale fusion, the cervical image segmentation network is solved and missed detection and misdetection problems in identifying precancerous cervical lesions is achieved, and the accurate segmentation and position judgment of cervical precancerous lesions and cervical mouth are achieved. , improves diagnostic accuracy.
Patent Information
- Application Number
- CN202411038739.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-15
AI Technical Summary
The existing cervical image segmentation network is prone to missed and missed detection when identifying precancerous cervical lesions, and fails to effectively judge the location of the lesion, affecting the accuracy of diagnosis.
Using cervical image processing method based on dual attention and multi-scale fusion, the optimal cervical image segmentation model is trained through deep learning algorithms to quickly identify and segment cervical precancerous lesions and cervical vents, and convert the segmentation results into lesion orientation information , provide clinical diagnostic support.
It significantly improves the diagnosis rate of precancerous cervical lesions, reduces the probability of missed and missed detection, provides accurate lesion location information, and supports biopsy and follow-up treatment.
Smart Images

Figure CN118967718A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a cervical image processing method based on dual attention and multi-scale fusion, belonging to the technical field of medical image intelligent processing. Background Art
[0002] Cervical cancer is the fourth most common cancer among women worldwide. Colposcopy can effectively improve the diagnosis rate of cervical precancerous lesions and is of great significance for preventing and maintaining women's health. Colposcopy is an important means of cervical cancer screening. It is used to observe the types of precancerous lesions in the cervix. Its clinical diagnostic value is higher than human papillomavirus (HPV) and ThinPrep cytology test (TCT). Clinicians diagnose the type of precancerous lesions as high-grade squamous intraepithelial lesions (HSIL) or low-grade squamous intraepithelial lesions (LSIL) through colposcopy. Through targeted treatment, most cervical precancerous lesions can be prevented from turning into cervical cancer. However, due to the small color difference between HSIL and LSIL and the similar color to the overall color of the cervical plane, the accuracy of colposcopy diagnosis depends on the subjective evaluation of clinicians and requires years of practical experience.
[0003] With the development of machine learning, some machine learning-based methods are applied to cervical image segmentation. Before classifying the lesion type, it is necessary to solve the influencing factors such as mirror reflection in the image, and then manually design the feature extractor according to the lesion characteristics. In recent years, convolutional neural networks have overcome the problem of manually designing features in machine learning by automatically learning the features in the image. More and more scholars have begun to use deep learning methods to classify cervical precancerous lesions in colposcopic images. However, most of the networks currently used for colposcopic image segmentation directly use traditional medical image segmentation networks, because the color difference between HSIL and LSIL is subtle, which is different from the pathological features of common medical images, which can easily cause missed detection and false detection during the segmentation process. At the same time, the current segmentation network for cervical images only performs segmentation and recognition on lesions, ignoring the need for doctors to determine the location of lesions in the clinical process. The location of the lesion can supplement the location information for subsequent biopsy or patient follow-up. Summary of the invention
[0004] The technical problem to be solved by the present invention is to overcome the defects of the prior art, provide a cervical image processing method based on dual attention and multi-scale fusion, realize the segmentation of precancerous lesion area and cervical os in colposcopy images based on deep learning algorithm, and can quickly identify and segment cervical precancerous lesions and cervical os, improve the diagnosis rate of precancerous lesions, and greatly reduce the probability of missed detection and false detection during the segmentation process. At the same time, according to the segmentation results of lesions and cervical os, simulate clinical diagnosis output, realize lesion orientation judgment, and provide lesion location information for biopsy and subsequent observation and treatment.
[0005] Preferentially, the present invention provides a cervical image processing method based on dual attention and multi-scale fusion, comprising: inputting a pre-acquired cervical image into an optimal cervical image segmentation model obtained by training, and outputting a segmentation mask of cervical precancerous lesions and cervical os;
[0006] The pixel position information of the segmented cervical precancerous lesions and cervical os is converted into relative orientation information.
[0007] Prioritize, train and obtain the optimal cervical image segmentation model, including:
[0008] Step 1: Preprocess the historical cervical images taken by clinicians during colposcopy to obtain training sets and test sets;
[0009] Step 2: Construct a cervical image segmentation model based on dual attention and multi-scale fusion;
[0010] Step 3: Train the cervical image segmentation model based on dual attention and multi-scale fusion to obtain the optimal cervical image segmentation model;
[0011] The optimal cervical image segmentation model is tested using the test set to determine whether the accuracy of the predicted label output by the optimal cervical image segmentation model is higher than the preset accuracy threshold. If so, the optimal cervical image segmentation model is determined to be qualified;
[0012] Among them, the predicted labels include the predicted high-grade squamous intraepithelial lesion segmentation mask, the low-grade squamous intraepithelial lesion segmentation mask, and the cervical os segmentation mask.
[0013] Preferably, step 1 comprises:
[0014] Historical cervical images of 170 different patients with high-grade squamous intraepithelial lesions and historical cervical images of different patients with low-grade squamous intraepithelial lesions were retrieved from the hospital information department, and the historical cervical images were annotated to obtain the true value labels including the real high-grade squamous intraepithelial lesion segmentation mask, the low-grade squamous intraepithelial lesion segmentation mask and the cervical os segmentation mask. The historical cervical images were scaled to convert the historical cervical images of different resolutions into historical cervical images of 224×224 size suitable for network input; the historical cervical images were divided into training set and test set in an 8:2 ratio, and the historical cervical images in the training set were rotated or mirrored to expand the number of historical cervical images in the training set and the test set.
[0015] Preferentially, the architecture of the constructed dual attention and multi-scale fusion based cervical image segmentation model includes an image encoder and an image decoder;
[0016] The encoder in the image encoder includes a first encoding stage, a second encoding stage, a third encoding stage and a fourth encoding stage; the first encoding stage includes a combination of a convolutional stem StemConv and a dual attention module; the second encoding stage, the third encoding stage and the fourth encoding stage have the same structure, all including patch embedding PatchMerging and a dual attention module.
[0017] Preferably, the structure of the convolutional stem StemConv contains a convolution operation with a stride of 2 and a convolution kernel of 3×3, a convolution operation with a stride of 1 and a convolution kernel of 3×3, two normalization layers and a GeLU activation function;
[0018] PatchMerging means: for a given input feature map , C is the number of channels of the image, H is the image height, W is the image width, and a convolution operation with a step size of O, padding of P, and a convolution kernel size of S is performed to obtain the output After that, the normalization process is performed, and the size W' and H' of the patch embedding output by PatchMerging are specifically expressed as:
[0019]
[0020] The dual attention module refers to: for a given input feature map The input feature map is averaged and pooled to obtain a vector; the average pooled vector is passed through the MLP network and the weight is adjusted to obtain a 1×1×C channel attention vector, which is then applied to the input feature map That is, each channel is multiplied by its own weight to obtain a new feature map
[0021] The feature map of size [C, H, W] is compressed using 1×1 convolution for channel compression, and then average pooling is used for spatial compression to obtain [C', H', W'];
[0022] Flatten each channel in the feature map of [C', H', W'] into a 1D vector to obtain a feature map of size [1, C', H', W'] after conversion;
[0023] Use a 1×8 convolution to multiply the feature map of [1,C',H',W'] to get the attention feature map;
[0024] Use concatenation to combine the obtained attention feature map with The feature maps are connected, and the connected feature maps are passed through the BN layer and the ReLU layer to obtain feature information.
[0025] Preferably, the image decoder includes a dual-branch fusion module and a convolution upsampling module, the dual-branch fusion module includes a first branch and a second branch, the first branch concatenates the feature information of the second encoding stage, the third encoding stage and the fourth encoding stage and puts them into a lightweight Hamburger for fusion processing;
[0026] The second branch puts the feature information of the second encoding stage, the third encoding stage and the fourth encoding stage into the multi-scale fusion module MSF for feature enhancement; the multi-scale fusion module MSF includes the C1BS convolution layer composed of conv1×1, BN layer, SiLU, the C3BS convolution layer composed of conv3×3, BN layer, SiLU and the cross-scale fusion convolution module CFC;
[0027] The specific implementation steps of the multi-scale fusion module MSF are as follows:
[0028] First, the feature information output from the fourth encoding stage passes through the C1BS convolution layer and is fused with the feature information output from the third encoding stage through a cross-scale fusion convolution module to obtain the fused feature A.
[0029] After the fusion feature A passes through the C1BS convolution layer, it is fused with the feature information output from the second encoding stage through the cross-scale fusion convolution module to obtain the fusion feature B;
[0030] After the fusion feature B passes through the C3BS convolution layer, it is fused with the feature information output from the third encoding stage through a cross-scale fusion convolution module to obtain the fusion feature C;
[0031] The fusion feature C passes through the C3BS convolution layer and the feature information obtained by the output of the fourth encoding stage after passing through the C1BS convolution layer is fused through the cross-scale fusion convolution module to obtain the fusion feature D;
[0032] Through the concatenation operation, the fusion features B, C and D are fused and output;
[0033] Among them, the cross-scale fusion convolution module concatenates the feature information of two different scales, and then adds the results after 1×1 convolution, 3×3 convolution, 7×7 convolution, and ReLu layer for further fusion operation;
[0034] The cross-scale fusion convolution module adds the result of the first branch and the result of the second branch and puts them into the convolution upsampling module; the convolution upsampling module includes a conv1x1 convolution layer and an upsampling layer;
[0035] The results of the first branch and the second branch are added and passed through conv1×1 to convert the number of channels of the feature map into the number of categories required for classification. The upsampling layer uses a bilinear interpolation upsampling method to restore the spatial resolution of the feature map passing through conv1×1 to the image resolution when it is input into the cervical image segmentation model based on dual attention and multi-scale fusion or the optimal cervical image segmentation model.
[0036] Preferably, the specific method of step 3 is as follows:
[0037] The historical cervical images in the prepared training set are placed in the cervical image segmentation model based on dual attention and multi-scale fusion. The cross entropy loss function and Dice loss function are used in the training process to measure the error between the true value label and the predicted label of the training set. The cross entropy loss function is used to evaluate the classification prediction of each pixel point in the segmentation process. The cross entropy loss function is specifically expressed as:
[0038]
[0039] Where N is the number of cervical image samples in the training set, C is the pixel value corresponding to category i, and the value range is [0,3]. i Refers to whether it is category i, if it is category i, then y i =1, otherwise y i =0; p i It refers to the probability that sample i belongs to C;
[0040] The Dice loss function is used to evaluate the difference between the predicted label and the true value label. It is obtained by subtracting the Dice similarity coefficient from 1, and the value range is [0, 1]. The Dice loss function is specifically expressed as:
[0041]
[0042] Among them, |X∩Y| represents the intersection of the real image and the predicted image, |X| and |Y| represent the number of elements respectively;
[0043] The total loss function of the cervical image segmentation model based on dual attention and multi-scale fusion is the sum of the cross entropy loss function and the Dice loss function. The total loss function is specifically expressed as:
[0044] L total =0.5×L C +0.5×L D ,
[0045] Finally, the weight with the highest prediction label accuracy during the training process is saved. When the test set is input, the optimal weight obtained by training the optimal cervical image segmentation model is loaded, and a segmentation test is performed on the test set data. The input historical cervical image is passed through the image encoder and image decoder in turn to obtain the segmentation mask of cervical precancerous lesions and cervical os.
[0046] Preferably, the pixel position information of the segmented cervical precancerous lesions and cervical os is converted into relative position information, and the specific method is as follows:
[0047] According to the pixel position information of the cervical precancerous lesions and the cervical os obtained by segmentation, the segmentation masks of the cervical precancerous lesions and the cervical os generated by the cervical image segmentation model based on dual attention and multi-scale fusion are binarized to obtain the contour information of the lesions and the cervical os;
[0048] According to the contour information of the cervical os, the pixel coordinates O(x,y) of the centroid of the cervical os are determined. The centroid coordinates are specifically expressed as:
[0049]
[0050] Among them, M 00 、M 01 and M 10 The calculation is specifically expressed as:
[0051]
[0052] Among them, M 00 is the zero-order moment, M 10 and M 01 is the first-order moment; I(x,y) is the pixel value of the historical cervical image at the coordinate (x,y); M 10 It means traversing every pixel in the historical cervical image (from coordinates (1,1) to (W,H)), multiplying the x coordinate of each pixel in the historical cervical image by its corresponding pixel value, and then adding up all the products; M 01 It means traversing each pixel in the historical cervical image (from coordinates (1,1) to (W,H)), multiplying the y coordinate of each pixel by its corresponding pixel value, and then adding up all the products to obtain the product accumulation result;
[0053] Then, according to the contour of the lesion in the historical cervical image, the coordinates of the two pixels with the longest distance in the lesion, p1(x1,y1) and p2(x2,y2);
[0054] Calculate the angle values A1 and A2 of p1, p2 and the centroid of the historical cervical image respectively, which can be expressed as:
[0055]
[0056] Taking the centroid as the origin of the coordinate axis, according to the angle range setting of the 12 o'clock orientation, the positions of the angle values A1 and A2 in the 12 o'clock orientation are determined, and the orientation range of the lesions in the historical cervical images is obtained.
[0057] Preferably, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the program.
[0058] Preferably, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.
[0059] The beneficial effects achieved by the present invention are:
[0060] The present invention realizes the segmentation of the precancerous lesion area and the cervical os in the colposcopy image based on the deep learning algorithm, and can quickly identify and segment the precancerous lesions and the cervical os, improve the diagnosis rate of precancerous lesions, and greatly reduce the probability of missed detection and false detection during the segmentation process. At the same time, according to the segmentation results of the lesions and the cervical os, the clinical diagnosis output is simulated to realize the judgment of the lesion orientation, and provide the lesion location information for biopsy and subsequent observation and treatment. Compared with the existing cervical image segmentation method, the present invention can fully integrate the global and local information, and improve the segmentation accuracy of the precancerous lesions and the cervical os.
[0061] After performing the segmentation task, the present invention converts the pixel positions of cervical lesions and cervical os into the 12-point orientations required by clinicians, which is convenient for biopsy and subsequent observation and treatment. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0063] Figure 1 is a flow chart of some embodiments of the present application;
[0064] Figure 2 is a schematic diagram of the structure of a cervical image segmentation model based on dual attention and multi-scale fusion in some embodiments of the present application;
[0065] Figure 3 is a schematic diagram of the structure of a dual attention module in some embodiments of the present application;
[0066] Figure 4is a schematic diagram of the structure of a multi-scale fusion module in some embodiments of the present application;
[0067] Figure 5 It is a schematic diagram of converting the pixel position information of the segmented cervical precancerous lesions and cervical os into relative orientation information in some embodiments of the present application. DETAILED DESCRIPTION
[0068] To facilitate the technical solution of the application, some concepts involved in the application are first explained below.
[0069] See also Figure 1 The present application discloses a cervical image processing method based on dual attention and multi-scale fusion, which is characterized by comprising: inputting a pre-acquired cervical image into an optimal cervical image segmentation model obtained by training, and outputting a segmentation mask of cervical precancerous lesions and cervical os; converting the pixel position information of the segmented cervical precancerous lesions and cervical os into relative orientation information.
[0070] The optimal cervical image segmentation model is trained, including:
[0071] Step 1: Preprocess the historical cervical images taken by clinicians during colposcopy to obtain training sets and test sets; the purpose of preprocessing is to screen and process the clinical images to obtain suitable images, which are then sent to the designed network for training and testing.
[0072] Step 2: Construct a cervical image segmentation model based on dual attention and multi-scale fusion;
[0073] Step 3: Train the cervical image segmentation model based on dual attention and multi-scale fusion to obtain the optimal cervical image segmentation model; use the test set to test the optimal cervical image segmentation model to determine whether the accuracy of the predicted label output by the optimal cervical image segmentation model is higher than the preset accuracy threshold. If so, the optimal cervical image segmentation model is considered qualified. The predicted labels include the predicted high-grade squamous intraepithelial lesion segmentation mask, the low-grade squamous intraepithelial lesion segmentation mask, and the cervical os segmentation mask. Cervical precancerous lesions include HSIL and LSIL, HSIL is a high-grade squamous intraepithelial lesion, and LSIL is a low-grade squamous intraepithelial lesion.
[0074] Step 4: Input the pre-acquired cervical image into the optimal cervical image segmentation model obtained through training, and output the segmentation mask of the cervical precancerous lesions and the cervical os; convert the pixel position information of the segmented cervical precancerous lesions and the cervical os into relative orientation information.
[0075] Step 1 The specific method is as follows: retrieve 170 historical cervical images of different patients with high-grade squamous intraepithelial lesions (HSIL) and low-grade squamous intraepithelial lesions (LSIL) from the hospital information department, annotate the historical cervical images, obtain the true value labels including the real high-grade squamous intraepithelial lesion segmentation mask, the real low-grade squamous intraepithelial lesion segmentation mask and the real cervical os segmentation mask, and scale the historical cervical images to convert the historical cervical images of different resolutions into 224×224 size historical cervical images suitable for network input; divide the historical cervical images into training sets and test sets at a ratio of 8:2, and rotate and mirror the historical cervical images in the training set to achieve data expansion. Specifically, first mark the correct segmentation mask of cervical precancerous lesions and cervical os on the historical cervical images, that is, the true value label, and compare them with the segmentation mask of cervical precancerous lesions and cervical os predicted by the later network training, that is, the predicted label. The annotated result label belongs to the accurate value, and the segmentation label output by the cervical image segmentation model based on dual attention and multi-scale fusion belongs to the predicted value. If the overlap between the true value label and the predicted label is higher, it proves that the network segmentation result designed by the present invention is closer to the accurate value, and the network segmentation result is effective and can provide auxiliary information for clinical diagnosis.
[0076] In the embodiment of the present application, the architecture of the cervical image segmentation model based on dual attention and multi-scale fusion is constructed, including an image encoder and an image decoder. The encoder includes four encoding stages, wherein the first encoding stage includes a combination of a convolutional stem StemConv and a dual attention module; the second encoding stage, the third encoding stage and the fourth encoding stage are the same, including patch embedding PatchMerging and a dual attention module; the structure of the convolutional stem StemConv includes a convolution operation with a step size of 2 and a convolution kernel of 3×3, a convolution operation with a step size of 1 and a convolution kernel of 3×3, two normalization layers and a GeLU activation function.
[0077] PatchMerging means: for a given input feature map C is the number of channels of the image, H is the image height, W is the image width, and a convolution operation with a step size of O, padding of P, and a convolution kernel size of S is performed to obtain the output After normalization, the patch embedding size W', H' output by PatchMerging is specifically expressed as:
[0078]
[0079] In the embodiment of the present application, the dual attention module refers to: for a given input feature map The input feature map is averaged and pooled to obtain a vector; the average pooled vector is passed through the MLP network and the weight is adjusted to obtain a 1×1×C channel attention vector, which is then applied to the input feature map That is, each channel is multiplied by its own weight to obtain a new feature map
[0080] The feature map of size [C, H, W] is compressed by 1×1 convolution, and then spatially compressed by average pooling to obtain [C', H', W']; each channel in the feature map of [C', H', W'] is flattened into a 1D vector to obtain a feature map of size [1, C', H', W'] after conversion; a 1×8 convolution is used to multiply the feature map of [1, C', H', W'] to obtain the attention feature map; the obtained attention feature map is concatenated with The feature maps are connected, and the connected feature maps are passed through the BN layer and the ReLU layer to obtain feature information.
[0081] In an embodiment of the present application, the image decoder includes a dual-branch fusion module and a convolution upsampling module, the dual-branch fusion module includes a first branch and a second branch, the first branch puts the feature information of the second encoding stage, the third encoding stage and the fourth encoding stage into a lightweight Hamburger for fusion processing after a concate operation; the second branch puts the feature information of the second encoding stage, the third encoding stage and the fourth encoding stage into a multi-scale fusion module MSF for feature enhancement; wherein the multi-scale fusion module MSF includes a C1BS convolution layer composed of conv1×1, BN layer, and SiLU, a C3BS convolution layer composed of conv3×3, BN layer, and SiLU, and a cross-scale fusion convolution module CFC.
[0082] In the embodiment of the present application, the specific implementation steps of the multi-scale fusion module MSF are as follows: first, the feature information output by the fourth encoding stage passes through the C1BS convolution layer, and is fused with the feature information output by the third encoding stage through a cross-scale fusion convolution module to obtain a fused feature A; after the fused feature A passes through the C1BS convolution layer, it is fused with the feature information output by the second encoding stage through a cross-scale fusion convolution module to obtain a fused feature B; after the fused feature B passes through the C3BS convolution layer, it is fused with the feature information output by the third encoding stage through a cross-scale fusion convolution module to obtain a fused feature C; after the fused feature C passes through the C3BS convolution layer, it is fused with the feature information output by the fourth encoding stage after the C1BS convolution layer through a cross-scale fusion convolution module to obtain a fused feature D; the fused feature B, the fused feature C and the fused feature D are finally fused and output through a concate operation.
[0083] In an embodiment of the present application, a cross-scale fusion convolution module concatenates feature information of two different scales, and adds the results after 1×1 convolution, 3×3 convolution, 7×7 convolution, and ReLu layer for further fusion operation; the cross-scale fusion convolution module adds the results of the first branch and the results of the second branch and puts them into a convolution upsampling module; wherein the convolution upsampling module includes a conv1x1 convolution layer and an upsampling layer; the results of the first branch and the results of the second branch are added and then used to convert the number of channels of the feature map into the number of categories required for classification through conv1×1. This step simultaneously completes feature extraction and category mapping, and then the upsampling layer uses a bilinear interpolation upsampling method to restore the spatial resolution of the feature map passed through conv1×1 to the image resolution of 224×224 when the cervical image segmentation model based on dual attention and multi-scale fusion is input.
[0084] In the embodiment of the present application, the specific method of step 3 is as follows: the historical cervical images in the prepared training set are placed in the cervical image segmentation model based on dual attention and multi-scale fusion; the cross entropy loss function and the Dice loss function are used in the training process to measure the error between the true value label and the predicted label of the training set. The cross entropy loss function is used to evaluate the classification prediction of each pixel point in the segmentation process. The cross entropy loss function is specifically expressed as:
[0085]
[0086] Where N is the number of cervical image samples in the training set, C is the pixel value corresponding to category i, and the value range is [0,3]. i Refers to whether it is category i, if it is category i, then y i =1, otherwise y i =0; p i It refers to the probability that sample i belongs to C;
[0087] The Dice loss function is used to evaluate the difference between the predicted label and the true label. It is obtained by subtracting the Dice similarity coefficient from 1, and its value range is [0, 1]. The Dice loss function is specifically expressed as:
[0088]
[0089] Among them, |X∩Y| represents the intersection of the real image and the predicted image, |X| and |Y| represent the number of elements respectively;
[0090] The total loss function of the cervical image segmentation model based on dual attention and multi-scale fusion is the sum of the cross entropy loss function and the Dice loss function. The total loss function is specifically expressed as:
[0091] L total=0.5×L C +0.5×L D
[0092] Finally, the weight with the highest prediction label accuracy during the training process is saved, and the optimal weight obtained from the training is loaded in the test part. The test set data is segmented and tested. The input historical cervical image is passed through the image encoder and image decoder in turn to obtain the segmentation mask of cervical precancerous lesions and cervical os.
[0093] In the embodiment of the present application, the specific method of step 4 is as follows:
[0094] Firstly, according to the pixel position information of the cervical precancerous lesions and cervical os obtained by segmentation, the segmentation masks of the cervical precancerous lesions and cervical os generated by the cervical image segmentation model based on dual attention and multi-scale fusion are binarized to obtain the contour information of the lesions and cervical os.
[0095] Secondly, according to the contour information of the cervical os, the pixel coordinates O(x,y) of the centroid of the cervical os are determined. The centroid coordinates are specifically expressed as:
[0096]
[0097] Among them, M 00 、M 01 and M 10 The calculation is specifically expressed as:
[0098]
[0099] Among them, M 00 is the zero-order moment, M 10 and M 01 is the first-order moment; I(x,y) is the pixel value of the historical cervical image at the coordinate (x,y), M 10 It means traversing every pixel in the historical cervical image (from coordinates (1,1) to (W,H)), multiplying the x coordinate of each pixel in the historical cervical image by its corresponding pixel value, and then adding up all the products; M 01 It means traversing each pixel in the historical cervical image (from coordinates (1,1) to (W,H)), multiplying the y coordinate of each pixel by its corresponding pixel value, and then adding up all the products to obtain the product accumulation result;
[0100] Then, according to the contour of the lesion in the historical cervical image, the coordinates of the two pixels with the longest distance in the lesion, p1(x1,y1) and p2(x2,y2);
[0101] Calculate the angle values A1 and A2 of p1, p2 and the centroid of the historical cervical image respectively, which can be expressed as:
[0102]
[0103] Then, taking the centroid as the origin of the coordinate axis, according to the angle range setting of the 12 o'clock orientation, the positions of angle value A1 and angle value A2 in the 12 o'clock orientation are determined, and the orientation range of the lesions in the historical cervical images is obtained. Finally, the lesion type and orientation results in the historical cervical images are output.
[0104] In an embodiment of the present application, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the program.
[0105] In an embodiment of the present application, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0106] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0107] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention invented herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not invented by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0108] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above are only specific implementation methods of the present application and are not used to limit the protection scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.
Claims
1. A cervical image processing method based on dual attention and multi-scale fusion, characterized in that: include: Input the pre-acquired cervical image into the trained optimal cervical image segmentation model, and output the segmentation mask of cervical precancerous lesions and cervical os; The pixel position information of the segmented cervical precancerous lesions and cervical os is converted into relative orientation information.
2. According to claim 1, a cervical image processing method based on dual attention and multi-scale fusion is characterized in that: The optimal cervical image segmentation model is trained, including: Step 1: Preprocess the historical cervical images taken by clinicians during colposcopy to obtain training sets and test sets; Step 2: Construct a cervical image segmentation model based on dual attention and multi-scale fusion; Step 3: Train the cervical image segmentation model based on dual attention and multi-scale fusion to obtain the optimal cervical image segmentation model; The optimal cervical image segmentation model is tested using the test set to determine whether the accuracy of the predicted label output by the optimal cervical image segmentation model is higher than the preset accuracy threshold. If so, the optimal cervical image segmentation model is determined to be qualified; Among them, the predicted labels include the predicted high-grade squamous intraepithelial lesion segmentation mask, the low-grade squamous intraepithelial lesion segmentation mask, and the cervical os segmentation mask.
3. According to claim 2, a cervical image processing method based on dual attention and multi-scale fusion is characterized in that: Step 1 includes: Historical cervical images of 170 different patients with high-grade squamous intraepithelial lesions and historical cervical images of different patients with low-grade squamous intraepithelial lesions were retrieved from the hospital information department, and the historical cervical images were annotated to obtain the true value labels including the real high-grade squamous intraepithelial lesion segmentation mask, the low-grade squamous intraepithelial lesion segmentation mask and the cervical os segmentation mask. The historical cervical images were scaled to convert the historical cervical images of different resolutions into historical cervical images of 224×224 size suitable for network input; the historical cervical images were divided into training set and test set in an 8:2 ratio, and the historical cervical images in the training set were rotated or mirrored to expand the number of historical cervical images in the training set and the test set.
4. The cervical image processing method based on dual attention and multi-scale fusion according to claim 3, characterized in that: The architecture of the constructed cervical image segmentation model based on dual attention and multi-scale fusion includes an image encoder and an image decoder; The encoder in the image encoder includes a first encoding stage, a second encoding stage, a third encoding stage and a fourth encoding stage; The first encoding stage includes a combination of convolutional stem StemConv and dual attention modules; the second, third and fourth encoding stages have the same structure, all including patch embedding PatchMerging and dual attention modules.
5. The cervical image processing method based on dual attention and multi-scale fusion according to claim 4, characterized in that: The structure of the convolutional stem StemConv contains a convolution operation with a stride of 2 and a convolution kernel of 3×3, a convolution operation with a stride of 1 and a convolution kernel of 3×3, two normalization layers and a GeLU activation function; PatchMerging means: for a given input feature map C is the number of channels of the image, H is the image height, W is the image width, and a convolution operation with a step size of O, padding of P, and a convolution kernel size of S is performed to obtain the output After normalization, the patch embedding size W', H' output by PatchMerging is specifically expressed as: The dual attention module refers to: for a given input feature map The input feature map is averaged and pooled to obtain a vector; the average pooled vector is passed through the MLP network and the weight is adjusted to obtain a 1×1×C channel attention vector, which is then applied to the input feature map That is, each channel is multiplied by its own weight to obtain a new feature map The feature map of size [C, H, W] is compressed using 1×1 convolution for channel compression, and then average pooling is used for spatial compression to obtain [C', H', W']; Flatten each channel in the feature map of [C', H', W'] into a 1D vector to obtain a feature map of size [1, C', H', W'] after conversion; Use a 1×8 convolution to multiply the feature map of [1, C', H', W'] to get the attention feature map; Use concatenation to combine the obtained attention feature map with The feature maps are connected, and the connected feature maps are passed through the BN layer and the ReLU layer to obtain feature information.
6. The cervical image processing method based on dual attention and multi-scale fusion according to claim 4, characterized in that: The image decoder includes a dual-branch fusion module and a convolution upsampling module. The dual-branch fusion module includes a first branch and a second branch. The first branch concatenates the feature information of the second encoding stage, the third encoding stage, and the fourth encoding stage and puts them into a lightweight Hamburger for fusion processing. The second branch puts the feature information of the second encoding stage, the third encoding stage and the fourth encoding stage into the multi-scale fusion module MSF for feature enhancement; The multi-scale fusion module MSF includes the C1BS convolution layer composed of conv1×1, BN layer, and SiLU, the C3BS convolution layer composed of conv3×3, BN layer, and SiLU, and the cross-scale fusion convolution module CFC; The specific implementation steps of the multi-scale fusion module MSF are as follows: First, the feature information output from the fourth encoding stage passes through the C1BS convolution layer and is fused with the feature information output from the third encoding stage through a cross-scale fusion convolution module to obtain the fused feature A. After the fusion feature A passes through the C1BS convolution layer, it is fused with the feature information output from the second encoding stage through the cross-scale fusion convolution module to obtain the fusion feature B; After the fusion feature B passes through the C3BS convolution layer, it is fused with the feature information output from the third encoding stage through a cross-scale fusion convolution module to obtain the fusion feature C; The fusion feature C passes through the C3BS convolution layer and the feature information obtained by the output of the fourth encoding stage after passing through the C1BS convolution layer is fused through the cross-scale fusion convolution module to obtain the fusion feature D; Through the concatenation operation, the fusion features B, C and D are fused and output; Among them, the cross-scale fusion convolution module concatenates the feature information of two different scales, and then adds the results after 1×1 convolution, 3×3 convolution, 7×7 convolution, and ReLu layer for further fusion operation; The cross-scale fusion convolution module adds the result of the first branch and the result of the second branch and puts them into the convolution upsampling module; the convolution upsampling module includes a conv1x1 convolution layer and an upsampling layer; The results of the first branch and the second branch are added and passed through conv1×1 to convert the number of channels of the feature map into the number of categories required for classification. Then the upsampling layer uses a bilinear interpolation upsampling method to restore the spatial resolution of the feature map passed through conv1×1 to the image resolution when it is input into the cervical image segmentation model based on dual attention and multi-scale fusion or the optimal cervical image segmentation model.
7. The cervical image processing method based on dual attention and multi-scale fusion according to claim 6, characterized in that: Step 3: The historical cervical images in the prepared training set are input into the cervical image segmentation model based on dual attention and multi-scale fusion. The cross entropy loss function and Dice loss function are used in the training process to measure the error between the true value label and the predicted label of the training set. The cross entropy loss function is used to evaluate the classification prediction of each pixel point in the segmentation process. The cross entropy loss function is specifically expressed as: Where N is the number of cervical image samples in the training set, C is the pixel value corresponding to category i, and the value range is [0,3]. i Refers to whether it is category i, if it is category i, then y i =1, otherwise y i =0; p i It refers to the probability that sample i belongs to C; The Dice loss function is used to evaluate the difference between the predicted label and the true value label. It is obtained by subtracting the Dice similarity coefficient from 1, and the value range is [0, 1]. The Dice loss function is specifically expressed as: Among them, |X∩Y| represents the intersection of the real image and the predicted image, |X| and |Y| represent the number of elements respectively; The total loss function of the cervical image segmentation model based on dual attention and multi-scale fusion is the sum of the cross entropy loss function and the Dice loss function. The total loss function is specifically expressed as: L total =0.5×L C +0.5×L D , Finally, the weight with the highest prediction label accuracy during the training process is saved. When the test set is input, the optimal weight obtained by training the optimal cervical image segmentation model is loaded, and a segmentation test is performed on the test set data. The input historical cervical image is passed through the image encoder and image decoder in turn to obtain the segmentation mask of cervical precancerous lesions and cervical os.
8. The cervical image processing method based on dual attention and multi-scale fusion according to claim 1, characterized in that: The pixel position information of the segmented cervical precancerous lesions and cervical os is converted into relative position information. The specific method is as follows: According to the pixel position information of the cervical precancerous lesions and the cervical os obtained by segmentation, the segmentation masks of the cervical precancerous lesions and the cervical os generated by the cervical image segmentation model based on dual attention and multi-scale fusion are binarized to obtain the contour information of the lesions and the cervical os; According to the contour information of the cervical os, the pixel coordinates O(x,y) of the centroid of the cervical os are determined. The centroid coordinates are specifically expressed as: Among them, M 00 、M 01 and M 10 The calculation is specifically expressed as: Among them, M 00 is the zero-order moment, M 10 and M O1 is the first-order moment; I(x,y) is the pixel value of the historical cervical image at the coordinate (x,y); M 10 It means traversing every pixel in the historical cervical image (from coordinates (1,1) to (W,H)), multiplying the x coordinate of each pixel in the historical cervical image by its corresponding pixel value, and then adding up all the products; M 01 It means traversing each pixel in the historical cervical image (from coordinates (1,1) to (W,H)), multiplying the y coordinate of each pixel by its corresponding pixel value, and then adding up all the products to obtain the product accumulation result; Then, according to the contour of the lesion in the historical cervical image, the coordinates of the two pixels with the longest distance in the lesion, p1(x1,y1) and p2(x2,y2); Calculate the angle values A1 and A2 of p1, p2 and the centroid of the historical cervical image respectively, which can be expressed as: Taking the centroid as the origin of the coordinate axis, according to the angle range setting of the 12 o'clock orientation, the positions of the angle values A1 and A2 in the 12 o'clock orientation are determined, and the orientation range of the lesions in the historical cervical images is obtained.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Cervical cancer pathological image cancerous tissue segmentation method and system based on Transform
CN119251250A
Cervical cell image analysis auxiliary method and system based on AI
CN120339228A
Multi-mode-based training method and system for cervical pathology image classification model
CN120808067A
