Cervical liquid-based cell detection and identification method and system based on deep learning

Abstract

The present invention belongs to the technical field of neural network image detection, and more specifically, relates to a method and system for cervical liquid-based cell detection and recognition based on deep learning. The method includes collecting cervical liquid-based cell image data and generating a sub-image set by cropping; labeling abnormal cells in the sub-image set of cervical liquid-based cell pathology images using a rectangular box as a bounding box; performing data enhancement processing on the labeled data to obtain a data set, and then dividing the data set into a training set, a validation set, and a test set; training, validating, and testing an improved YOLOv5 network model using the data set to obtain a trained improved YOLOv5 network model; and inputting the cervical liquid-based cell pathology image to be analyzed into the trained improved YOLOv5 network model for recognition. The present invention solves the problems of missed detection and false detection in the prior art, as well as the insufficient accuracy and robustness of target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural network image detection, and more specifically, to a method and system for cervical liquid-based cell detection and identification based on deep learning. Background Art

[0002] Cervical cancer, the most common gynecological malignancy, has an increasing incidence and mortality rate, with a younger prevalence. This poses a serious threat to women's health and well-being, making early prevention and treatment crucial. Currently, the cervical thin-layer cytology test (TCT) is an important tool for cervical cancer screening. Early detection of abnormal cervical cells during cervical cancer screening increases the chances of timely treatment. The TCT test uses a New Bicycle automated cytometer to prepare a 2cm diameter thin-layer cytology image. The sample is wet-fixed with 95% alcohol and then examined under a microscope. It is negative for intraepithelial lesions (IELs) and positive for other lesions, including squamous cell carcinoma (SCC), low-grade squamous intraepithelial lesions (LSIL), atypical squamous cell lesions (ASCH), high-grade squamous intraepithelial lesions (HSIL), and atypical glandular epithelium (AGC). Microscopic screening of cervical smears by cytologists is a widely used method in routine examinations, consuming a significant amount of cytologist time and effort. Technically, cytologists must detect only a few abnormal and malignant cells in a sample consisting of tens of thousands of cells. Furthermore, due to the lack of experienced and qualified cytologists and cytotechnologists, as well as factors such as their diagnostic experience, emotions, and fatigue, human factors can lead to data misinterpretation. Computerized cytological analysis appropriately addresses this urgent need, reducing the workload of cytologists and the potential for misdiagnosis. Therefore, automated assisted screening based on cervical cytology has become necessary.

[0003] Chinese invention patent CN112380900A discloses a method and system for classifying digital images of uterine fluid-based cells based on deep learning. The method includes: an image transmission module, a detection module and a film reading module. The image transmission module is used to transmit the digital image of cervical fluid-based cells to be identified to the detection module; the detection module is used to detect various types of cells in the digital image of cervical fluid-based cells based on the trained Faster-RCNN model and obtain detection results; the film reading module is used to review the detection results.

[0004] However, there are the following objective difficulties in deep learning-assisted cervical cytology testing.

[0005] Due to the privacy of doctors and patients, there are few public cervical cell data sets, and the quality is uneven, and the cell types are limited, while deep learning requires a large amount of data; secondly, cervical cancer cells are very complex. At the current stage, even if some methods can be used to conclude the condition, due to the particularity of the industry, doctors still need to observe and confirm again. Providing doctors with more accurate reference opinions and saving time for re-judgment are the key points, while the existing technology has a low detection speed and there are many cases of missed detection and false detection; in the entire cervical cell image, there are more complex components, and generally image preprocessing methods are required to better utilize neural networks for recognition training, but it takes a long time, and the image transformation after preprocessing is not smooth and natural enough. Summary of the Invention

[0006] The present invention aims to overcome at least one of the above-mentioned defects of the prior art and provide a cervical liquid-based cell detection and identification method and system based on deep learning to solve the problems of missed detection and false detection in the prior art, as well as the insufficient accuracy and robustness of target detection.

[0007] The detailed technical solutions of the present invention are as follows:

[0008] To achieve the above objectives, the present invention provides a method for detecting and identifying cervical liquid-based cells based on deep learning, comprising the following steps:

[0009] S1, collecting cervical liquid-based cell image data and generating a sub-image set by cropping;

[0010] S2, using a rectangular box as a bounding box to mark abnormal cells in a sub-image set of cervical liquid-based cytopathology images;

[0011] The abnormal cell annotation information is saved in txt format, including the category number and center point coordinates of the annotation bounding box, as well as the width and height of the bounding box. The center point coordinates, width, and height of the rectangular box are normalized.

[0012] S3. Perform data augmentation on the labeled data to obtain a dataset, which is then divided into a training set, a validation set, and a test set.

[0013] S4. Train, verify, and test the improved YOLOv5 network model using the data set to obtain a trained improved YOLOv5 network model.

[0014] The improved YOLOv5 network model includes a backbone network, a feature fusion module and a head network; a deformable convolution ODConv module is added after the upsampling operation of the feature fusion module;

[0015] S5. Input the cervical liquid-based cytopathology image to be analyzed into the trained improved YOLOv5 network model for recognition.

[0016] Furthermore, the upsampling method of the feature fusion module of the improved YOLOv5 network model adopts an improved bilinear interpolation method, such as Figure 6 As shown, the details are as follows:

[0017] Get four adjacent pixel points Q in the bounding box 11 ,Q 21 ,Q 12 ,Q 22 , four adjacent pixels are distributed in two rows and two columns, the X axis represents the horizontal direction of the image, the Y axis represents the vertical direction of the image, the origin of the coordinate axis (0,0) is in the lower left corner of the image, the X axis extends to the right as positive, and the Y axis extends upward as positive;

[0018] Four adjacent pixels are first linearly transformed on the X-axis to obtain the interpolated pixel value of point R in each row of pixels in the image. Point R1 is the supplementary interpolated pixel point obtained by linearly transforming the adjacent Q points in the first row on the X-axis. Point R2 is the supplementary interpolated pixel point obtained by linearly transforming the adjacent Q points in the second row on the X-axis.

[0019]

[0020]

[0021] The calculation of point P in the internal area surrounded by four adjacent pixels is as follows:

[0022]

[0023] x is the coordinate value of point P on the X axis. Point P represents the pixel interpolation point obtained by linearly transforming the two pixel points R1 and R2 on the Y axis. x1 is the coordinate value of Q 11 , Q 12 The horizontal coordinate value of x2 is Q 21 , Q 22 The horizontal coordinate value of y1 is Q 11 ,,Q 21 The vertical coordinate value of y2 is Q 12 , Q 22 The vertical coordinate value of point P is x and y, respectively;

[0024] Summarize the above content into the formula to be calculated f(x, y), and get point P (x, y):

[0025]

[0026] Calculate the position of point P and Q 11 , Q 21 , Q 12 , Q22 The Euclidean distance ρ is calculated as follows:

[0027]

[0028] The x and y in the formula are the horizontal and vertical coordinate values ​​of point P respectively. q 、y q Substitute the horizontal and vertical coordinate values ​​of each Q point.

[0029] Select the point Q farthest from point P as point D, and then perform linear interpolation;

[0030] Linear interpolation is performed between point D and point P. The formula is:

[0031]

[0032]

[0033] f(T)=af(D)+(1-a)f(P) (8);

[0034] In the above formula, f(D) represents the pixel value of point D, f(P) represents the pixel value of point P, xd represents the horizontal coordinate value of point D, y d represents the vertical coordinate value of point D, x is the horizontal coordinate value of point P, y is the vertical coordinate value of point P, x t Represents the horizontal coordinate value of point T, y t represents the vertical coordinate value of point T, a represents the weight, and f(T) is the calculated pixel value of the insertion point T, where points D and P are the interpolation points.

[0035] The default upsampling method used in the original YOLOv5 model is the nearest neighbor interpolation algorithm. When the image is enlarged, the missing pixels are directly generated by using the nearest original pixel value. The enlarged image has severe mosaics, and the reduced image has severe distortion. Therefore, the improved bilinear interpolation method is used in this invention to replace the nearest neighbor interpolation algorithm.

[0036] Furthermore, in the head network of the improved YOLOv5 network model, an improved non-maximum suppression algorithm is used to suppress redundant bounding boxes during detection:

[0037] S41: Set the confidence threshold and IOU threshold of the bounding box;

[0038] S42: Generate a candidate box list by sorting the bounding boxes in descending order according to the confidence level;

[0039] S43: Calculate the average overlap of the candidate box list;

[0040] S44: Determine whether the average overlap ratio is greater than a rated overlap ratio, wherein the rated overlap ratio is preferably 0.5;

[0041] S45: When the average overlap is less than 0.5, select the candidate box A with the highest confidence and add it to the output list, and delete it from the candidate box list;

[0042] Then calculate the IOU value between candidate box A and all candidate boxes in the candidate box list, delete the candidate boxes with a value greater than the IOU threshold, and retain the candidate boxes with a value less than the IOU threshold;

[0043] S46: When the average overlap is ≥ 0.5, select the box A with the highest confidence and add it to the output list, and delete it from the candidate box list;

[0044] Then calculate the IOU value of candidate box A with all boxes in the candidate box list. For candidate boxes with a confidence score higher than the confidence threshold, select candidate boxes with a confidence score higher than the confidence threshold, reduce their confidence scores by reducing the score strategy, and retain them; retain candidate boxes with a confidence score lower than the IOU threshold.

[0045] The strategy for reducing the bounding box score in Soft-NMS is:

[0046]

[0047]

[0048] S47: Repeat the above process until all candidate boxes are processed.

[0049] Furthermore, the calculation of the average overlap is specifically as follows:

[0050] S431: Calculate the overlapping area between the selected candidate box and all other candidate boxes;

[0051] S432: Calculate the ratio of the overlapping area of ​​the selected candidate box and all other candidate boxes to the area of ​​the selected candidate box itself;

[0052] S433: summing up the overlap rates of all candidate frames in the candidate frame list and dividing the sum by the total number of candidate frames to obtain an average overlap rate;

[0053] The specific formula for calculating the average overlap rate is as follows:

[0054]

[0055] Among them, overlap_area represents the overlapping area between target candidate boxes, box_area represents the area of ​​the target candidate box, and N represents the total number of target candidate boxes.

[0056] This embodiment also provides a cervical liquid-based cytopathology image detection and recognition system based on deep learning, which includes the following functional modules: medical image cropping module, image annotation module, data partitioning module, image enhancement module, network training module, and target detection module:

[0057] Medical image cropping module: crops the collected cervical liquid-based cell images;

[0058] Image annotation module: In the system's image annotation window, you can annotate the image data by uploading the image to the system, selecting the directory to store the annotation file, and then annotating the image until all the image data is annotated;

[0059] Data partitioning module: In the system's partitioning data set window, by selecting the file directory corresponding to the data set in the system and the file directory where the annotation file is located, the image data set is randomly divided into training set, validation set, and test set. The partition ratio can be customized and stored in different file directories;

[0060] Image enhancement module: In the system's image enhancement window, image enhancement is performed. The image enhancement methods that can be selected in the system of the present invention include: inscribed circle rotation image enhancement, image quartering enhancement, Gaussian filtering, image rotation, and grayscale histogram equalization. After selecting an image to upload to the system, different image enhancement methods can be selected to view the effect of the image enhancement; batch data enhancement can also be performed. In the window, select the file directory path for storing the image, then select the file directory path for storing the image annotation, select the image file directory to be exported after enhancement, and the storage location of the exported annotation file after image enhancement. Batch image enhancement is performed according to the enhancement method, and the enhanced image annotation information will also change accordingly;

[0061] Network training module: In the system's detection window, the cervical liquid-based cell pathology images are detected. First, select the weight file, which is obtained after the YOLOv5 network training is completed. Import the weight file in the system's detection window; select the image to be detected, and click the Detect button after selection to identify the type of abnormal cells. The lesion area can be framed with a rectangular frame, and the probability of identifying it as an abnormal cell of this type can be displayed. The system can automatically count the number of abnormal cells and display it on the system display page; you can also choose batch detection, select the file directory where the test image is located, select the storage directory for exporting the detection image file, and finally click the Detect button to perform batch image detection;

[0062] Target detection module: After batch detection of images, the system displays the number of abnormal cell categories in all test images on the page and automatically performs statistics.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) The present invention provides a method for detecting and recognizing cervical liquid-based cytopathology images based on deep learning, improves the non-maximum suppression (NMS) algorithm, and determines whether to use NMS or Soft-NMS for target suppression based on the overlap of categories, which can retain the diversity of targets. When using NMS, the overlap rate between targets is high, and the target with the highest confidence can be selected to reduce redundant candidate frames; when using Soft-NMS, more targets are retained by reducing the confidence of overlapping candidate frames, thereby improving target diversity; reducing missed detections and false detections, and suppressing according to the overlap rate of categories can help reduce missed detections and false detections; by selecting an appropriate suppression method based on the overlap rate of categories, the accuracy and robustness of target detection can also be improved; and it can be adjusted and adapted according to different scenarios and target characteristics.

[0065] (2) The deep learning-based cervical fluid-based cytopathology image detection and recognition method provided by the present invention is trained based on an improved YOLOv5 deep learning network. The YOLOv5 deep learning network performs well in target detection tasks, has high accuracy and detection speed, and can be applied to various target detection scenarios. In the present invention, deformable convolution is introduced into the original YOLOv5 deep learning network, thereby improving the network's accuracy for this data set.

[0066] (3) The deep learning-based cervical fluid-based cell detection and recognition method and system provided by the present invention use improved bilinear interpolation for image upsampling in the feature fusion module during model training, which can smoothly amplify features, maintain the details of image features, and reduce jagged or blocky shadows. Compared with nearest neighbor interpolation and original bilinear interpolation, it can provide smoother and more natural transformation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is the overall process of the method of the present invention.

[0068] Figure 2 This is a system module diagram described in Example 1 of the present invention.

[0069] Figure 3 This is a diagram of the improved YOLOv5 network model in Example 1 of the present invention.

[0070] Figure 4 4 is a flow chart of the improved non-maximum suppression algorithm in Example 1 of the present invention.

[0071] Figure 5 This is a rendering of the improved bilinear interpolation method in Example 1 of the present invention.

[0072] Figure 6 This is a point diagram of the improved bilinear interpolation method in Example 1 of the present invention. DETAILED DESCRIPTION

[0073] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0074] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0075] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0076] In the absence of conflict, the embodiments of the present disclosure and the features thereof may be combined with each other.

[0077] Example 1

[0078] This embodiment provides a method for detecting and identifying cervical liquid-based cell pathology images based on deep learning, such as Figure 1 As shown:

[0079] S1, collecting cervical liquid-based cell image data and generating a sub-image set by cropping;

[0080] The present invention classifies and identifies cervical liquid-based pathology images. Data is first collected, and abnormal cells are manually marked on cervical cytology digital pathology images in collaboration with hospitals and other authoritative pathology experts to generate image resources for artificial intelligence deep learning. The pixels of cervical liquid-based cell full-field pathology digital slice images are generally around tens of thousands by tens of thousands of pixels. Slices that are too large cannot be directly fed into the model for training. A cropping operation is required to crop the slice image into an image area of ​​1000×1000 pixels. The cervical liquid-based cell pathology digital slices are cropped to generate a sub-image set.

[0081] S2, using a rectangular box as a bounding box to mark abnormal cells in a sub-image set of cervical liquid-based cytopathology images;

[0082] The cervical liquid-based cytopathology images are annotated. Six types of abnormal cells are annotated in the system of the present invention, namely low-grade squamous intraepithelial lesion cells (LSIL), atypical squamous cells of undetermined significance (ASC-US), high-grade squamous intraepithelial lesion (HSIL), atypical glandular cells (AGC-NOS), uterine adenocarcinoma cells (ADE), and clue cells (BO). The target position is marked with a rectangular frame, and the abnormal cell annotation information is saved in txt format. The saved content includes the category number and center point coordinates of the annotated rectangular frame, as well as the width and height of the rectangular frame. The center point coordinates, width, and height of the rectangular frame are normalized, and the file name is consistent with the image name.

[0083] S3. Perform data augmentation on the labeled data to obtain a dataset, which is then divided into a training set, a validation set, and a test set:

[0084] The system of the present invention applies a method for enhancing small-sample image data for cervical liquid-based cytopathology detection, which can automatically augment the labeled image data set, increasing the number of images in the original data set exponentially. The enhanced images can meet the training requirements of the deep learning network and improve the network model's recognition and generalization learning capabilities for cervical cancer cells.

[0085] S4. Train, verify and test the improved YOLOv5 network model through the data set to obtain the trained improved YOLOv5 network model, such as Figure 3 As shown;

[0086] The improved YOLOv5 network model includes a backbone network, a feature fusion module and a head network; the backbone network is used for feature extraction; the feature fusion module is used to fuse features of different scales, and can realize parameter aggregation of features of different detection layers from different backbone layers to obtain predicted feature maps; in the head network, NMS is used to suppress redundant boxes during non-maximum suppression detection;

[0087] Specifically, a deformable convolution ODConv module is added after the upsampling operation of the feature fusion module.

[0088] The traditional convolution operation is to slide a fixed-size convolution kernel on the input image and perform a weighted sum of the local receptive fields at each location. However, in some cases, the fixed-size convolution kernel cannot adapt well to the complex shapes and spatial variations in the image;

[0089] The Deformable Convolution ODConv module offsets the convolution kernel by introducing additional learnable parameters, so that the shape of the convolution kernel can dynamically adapt to changes in the input image. The Deformable Convolution ODConv module offsets the sampling points of each convolution kernel from the original position to a new position, which is learned by the network;

[0090] The Deformable Convolution (ODConv) module can model local features using deformations, thereby better capturing the shape changes and details of the target. While traditional convolution operations can only sample fixed locations within the receptive field, the Deformable Convolution (ODConv) module can flexibly sample based on the specific shape of the target, improving feature representation capabilities.

[0091] Furthermore, the upsampling method of the feature fusion module of the improved YOLOv5 network model adopts an improved bilinear interpolation method, such as Figure 6 As shown, the details are as follows:

[0092] Get four adjacent pixel points Q in the bounding box 11 , Q 21 , Q 12 , Q 22 , four adjacent pixels are distributed in two rows and two columns. The X axis represents the horizontal direction of the image, and the Y axis represents the vertical direction of the image. The origin of the coordinate axis (0, 0) is in the lower left corner of the image. The X axis extends to the right as positive, and the Y axis extends upward as positive. Each pixel has a unique position in the coordinate system and is represented by an ordered coordinate pair;

[0093] The four adjacent pixels are first linearly transformed on the X-axis to obtain the interpolated pixel value of point R in each row of pixels in the image. Point R1 is the supplementary interpolated pixel point obtained by linearly transforming the adjacent Q points in the first row on the X-axis: Point R2 is the supplementary interpolated pixel point obtained by linearly transforming the adjacent Q points in the second row on the X-axis:

[0094]

[0095]

[0096] In the formula, x is the coordinate value of point P on the X axis, point P represents the pixel interpolation point obtained by performing a linear transformation on the Y axis on the two pixel points R1 and R2, and x1 is the Q 11 , Q 12 The horizontal coordinate value of x2 is Q 21 , Q 22 The horizontal coordinate value of y1 is Q 11 ,,Q 21 The vertical coordinate value of y2 is Q 12 , Q 22The vertical coordinate value of point P is x and y, respectively;

[0097] This area (between four adjacent pixels Q 11 ,,Q 21 , Q 12 , Q 22 The P point of the inner area enclosed by is calculated as follows:

[0098]

[0099] Summarize the above content into the formula to be calculated f(x, y), and get point P (x, y):

[0100]

[0101] Calculate the position of point P and Q 11 , Q 21 , Q 12 , Q 22 The Euclidean distance ρ is calculated as follows:

[0102]

[0103] The x and y in the formula are the horizontal and vertical coordinate values ​​of point P respectively. q 、y q Substitute the horizontal and vertical coordinate values ​​of each Q point;

[0104] Select the point Q farthest from point P as point D and perform linear interpolation again;

[0105] Linear interpolation is performed between point D and point P. The formula is:

[0106]

[0107]

[0108] f(T)=af(D)+(1-a)f(P) (8);

[0109] In the above formula, f(D) represents the pixel value of point D, f(P) represents the pixel value of point P, and x d Represents the horizontal coordinate value of point D, y d represents the vertical coordinate value of point D, x is the horizontal coordinate value of point P, y is the vertical coordinate value of point P, x t Represents the horizontal coordinate value of point T, y t represents the vertical coordinate value of point T, a represents the weight, f(T) is the calculated pixel value of the insertion point T, where points D and P are the interpolation points;

[0110] The default upsampling method used in the original YOLOv5 model is the nearest neighbor interpolation algorithm. When the image is enlarged, the missing pixels are directly generated by using the nearest original pixel value. The enlarged image has severe mosaics, and the reduced image has severe distortion. In this invention, an improved bilinear interpolation method is used to replace the nearest neighbor interpolation algorithm.

[0111] Compared with the common bilinear interpolation, the improved bilinear interpolation algorithm of the present invention has the following advantages: Figure 5 As shown:

[0112] It can provide higher interpolation quality, reconstruct image details more accurately, and reduce the appearance of artifacts and jagged edges; the traditional bilinear interpolation method may cause edge blur or loss of details when the image is enlarged, while the improved bilinear interpolation algorithm can better maintain edge information, reduce the edge blur problem when the image is enlarged, and thus improve the visual quality of the image.

[0113] The improved bilinear interpolation algorithm considers more pixel information to better estimate the value of the target pixel; the improved bilinear interpolation algorithm can better maintain the brightness of the image, and can improve the brightness of the traditional bilinear interpolation sampling by 2-3%.

[0114] Traditional bilinear interpolation methods may produce jagged edges or artifacts when the image is magnified, which affects the smoothness of the image. The improved algorithm uses more complex interpolation calculations to more accurately reconstruct image details, reduce the appearance of jagged edges, and improve image smoothness by 30-40%. The deformable convolution ODConv module can adaptively adjust the shape of the receptive field and the position of the sampling points to adapt to the deformation and position changes of the target. Combined with the above-mentioned improved upsampling method, more detailed deformation modeling can be performed in the high-resolution feature map, improving the ability to perceive target deformation, which helps to improve the accuracy and robustness of the target detection model for targets of different scales and shapes.

[0115] Furthermore, an improved non-maximum suppression algorithm is used in the head network of the improved YOLOv5 network model to suppress redundant bounding boxes during detection, such as Figure 4 As shown, specifically:

[0116] S41: Set the confidence threshold and IOU threshold of the bounding box;

[0117] S42: Generate a candidate box list by sorting the bounding boxes in descending order according to the confidence level;

[0118] S43: Calculate the average overlap of the candidate box list;

[0119] S44: Determine whether the average overlap ratio is greater than a rated overlap ratio, wherein the rated overlap ratio is preferably 0.5;

[0120] S45: When the average overlap is less than 0.5, select the candidate box A with the highest confidence and add it to the output list, and delete it from the candidate box list;

[0121] Then calculate the IOU value between candidate box A and all candidate boxes in the candidate box list, delete the candidate boxes with a value greater than the IOU threshold, and retain the candidate boxes with a value less than the IOU threshold;

[0122] S46: When the average overlap is ≥ 0.5, select the box A with the highest confidence and add it to the output list, and delete it from the candidate box list;

[0123] Then calculate the IOU value of candidate box A with all boxes in the candidate box list. For candidate boxes with a confidence score higher than the confidence threshold, select candidate boxes with a confidence score higher than the confidence threshold, reduce their confidence scores by reducing the score strategy, and retain them; retain candidate boxes with a confidence score lower than the IOU threshold.

[0124] For candidate boxes whose overlap rate is higher than a predefined threshold, reduce their scores; the strategy for reducing the score of bounding boxes in Soft-NMS is:

[0125]

[0126]

[0127] In the above formula, S i is the confidence score of each candidate box, b i Represents each candidate box, M represents the candidate box with the highest score in the candidate box list, and D represents all candidate boxes minus the candidate boxes in the saved candidate box list.

[0128] S47: Repeat the above process until all candidate boxes are processed.

[0129] The average overlap ratio is a commonly used metric to measure the average degree of overlap between objects in object detection tasks. It calculates the ratio of the overlapping area between the object bounding boxes to the total area of ​​the object bounding boxes and takes the average value. Furthermore, the calculation of the average overlap ratio is specifically:

[0130] S431: Calculate the overlapping area between the selected candidate box and all other candidate boxes;

[0131] S432: Calculate the ratio of the overlapping area of ​​the selected candidate box and all other candidate boxes to the area of ​​the selected candidate box itself;

[0132] S433: summing up the overlap rates of all candidate frames in the candidate frame list and dividing the sum by the total number of candidate frames to obtain an average overlap rate;

[0133] The specific formula for calculating the average overlap rate is as follows:

[0134]

[0135] Among them, overlap_area represents the overlapping area between target candidate boxes, box_area represents the area of ​​the target candidate box, and N represents the total number of target candidate boxes.

[0136] S5. Input the cervical liquid-based cytopathology image to be analyzed into the trained improved YOLOv5 network model for recognition.

[0137] This embodiment also provides a cervical liquid-based cytopathology image detection and recognition system based on deep learning, such as Figure 2 As shown, it includes medical image cropping module, image annotation module, data division module, image enhancement module, network training module, and target detection module:

[0138] Medical image cropping module: crops the collected cervical liquid-based cell images.

[0139] Image annotation module: Annotate image data by uploading the image to the system, selecting the directory to store the annotation file, and then annotating the image until all the image data is annotated.

[0140] Data Partitioning Module: In the Dataset Partitioning window, select the file directory corresponding to the dataset and the directory where the annotation files are located in the system to randomly divide the image dataset into training, validation, and test sets. The partitioning ratio can be customized and stored in different file directories.

[0141] Image enhancement module: In the system's image enhancement window, image enhancement is performed. The image enhancement methods that can be selected in the system of the present invention include: inscribed circle rotation image enhancement, image quartering enhancement, Gaussian filtering, image rotation, and grayscale histogram equalization. After selecting an image to upload to the system, you can select different image enhancement methods to view the effect after image enhancement; you can also perform batch data enhancement. In the window, select the file directory path for storing the image, then select the file directory path for storing the image annotations, select the image file directory to be exported after enhancement, and select the storage location of the exported annotation files after image enhancement. Batch image enhancement is performed according to the enhancement method, and the enhanced image annotation information will also change accordingly.

[0142] Network training module: The training set, validation set, and test set are input into the improved YOLOv5 network model for model training.

[0143] Target detection module: First, select the weight file in the system's detection window. The weight file is obtained after the YOLOv5 network training is completed. Import the weight file in the system's detection window; select the image to be detected, and click the detection button after selection to identify the type of abnormal cells. The lesion area can be framed with a rectangular frame, and the probability of identifying it as an abnormal cell of this type can be displayed. The system can automatically count the number of abnormal cells and display it on the system display page. You can also choose batch detection, select the file directory where the test image is located, select the storage directory for the exported test image file, and finally click the detection button to perform batch image detection. The number of abnormal cell categories in all test images will be displayed on the system display page, and statistics will be automatically performed.

[0144] The following is the accuracy comparison result of the present invention:

[0145] Comparison chart of the accuracy and average precision of the original model before improvement:

[0146]

[0147] It can be seen from the above table that the improved YOLOv5 model of the present invention has improved both the accuracy and recall rates in the detection of cervical fluid-based abnormal cell pathology images, with the accuracy rate increased by 4.6% and the recall rate increased by 0.7%.

[0148] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting and identifying cervical liquid-based cells based on deep learning, characterized in that: include; S1, collecting cervical liquid-based cell image data and generating a sub-image set by cropping; S2, using a rectangular box as a bounding box to mark abnormal cells in a sub-image set of cervical liquid-based cytopathology images; The abnormal cell annotation information is saved in txt format, including the category number and center coordinates of the annotated rectangular box, as well as the width and height of the bounding box. The center coordinates, width, and height of the bounding box are normalized. S3. Perform data augmentation on the labeled data to obtain a dataset, which is then divided into a training set, a validation set, and a test set. S4. Train, verify, and test the improved YOLOv5 network model using the data set to obtain a trained improved YOLOv5 network model. The improved YOLOv5 network model includes a backbone network, a feature fusion module and a head network; a deformable convolution ODConv module and an auxiliary detection decoupling head are added after the upsampling operation of the feature fusion module; S5. Inputting the cervical liquid-based cytopathology image to be analyzed into the trained improved YOLOv5 network model for recognition; The upsampling method of the feature fusion module of the improved YOLOv5 network model adopts an improved bilinear interpolation method, which is as follows: Get four adjacent pixels in the bounding box In the area composed of four adjacent pixels, there are two rows and two columns. First, a linear transformation is performed on the X-axis to find the pixel point R in each row: （1）； （2）； Perform another linear transformation on the Y axis to find point P in the area: （3）； Point P represents the pixel interpolation point obtained by performing a linear transformation on the Y axis on the two pixel points R1 and R2. for 、 The horizontal axis value of for 、 The horizontal axis value of for 、 The vertical coordinate value of for 、 The vertical coordinate value of ; Summarized into the calculation formula f(x,y): （4）； The position of point P is calculated by The Euclidean distance of Select the point Q farthest from point P as point D, and then perform linear interpolation; Linear interpolation is performed between point D and point P. The formula is: （6） 1- （7） （8）； Where, represents the pixel value of point D, represents the pixel value of point P, represents the horizontal coordinate value of point D, Represents the vertical coordinate value of point D, is the horizontal coordinate value of point P, is the vertical coordinate value of point P, Represents the horizontal coordinate value of point T, Represents the vertical coordinate value of point T, represents the weight, To calculate the pixel value of the insertion point T, point D and point P are the interpolation points.

2. The method for detecting and identifying cervical liquid-based cells based on deep learning according to claim 1, characterized in that: In the head network of the improved YOLOv5 network model, the improved non-maximum suppression algorithm is used to suppress redundant bounding boxes during detection: S41: Set the confidence threshold and IOU threshold of the bounding box; S42: Generate a candidate box list by sorting the bounding boxes in descending order according to the confidence level; S43: Calculate the average overlap of the candidate box list; S44: Determine whether the average overlap ratio is greater than a rated overlap ratio, wherein the rated overlap ratio is 0.5; S45: When the average overlap is less than the rated overlap, select the candidate box A with the highest confidence and add it to the output list, and delete it from the candidate box list; Then calculate the IOU value between candidate box A and all candidate boxes in the candidate box list, delete the candidate boxes with a value greater than the IOU threshold, and retain the candidate boxes with a value less than the IOU threshold. S46: When the average overlap is greater than or equal to the rated overlap, select the box A with the highest confidence and add it to the output list, and delete it from the candidate box list; Then calculate the IOU value of candidate box A with all boxes in the candidate box list. For candidate boxes with a confidence score higher than the confidence threshold, select candidate boxes with a confidence score higher than the confidence threshold, reduce their confidence scores by reducing the score strategy, and retain them; retain candidate boxes with a confidence score lower than the IOU threshold. S47: Repeat the above process until all candidate boxes are processed.

3. The method for detecting and identifying cervical liquid-based cells based on deep learning according to claim 2, characterized in that: The calculation of the average overlap is specifically as follows: S431: Calculate the overlapping area between the selected candidate box and all other candidate boxes; S432: Calculate the ratio of the overlapping area of ​​the selected candidate box and all other candidate boxes to the area of ​​the selected candidate box itself; S433: summing up the overlap rates of all candidate frames in the candidate frame list and dividing the sum by the total number of candidate frames to obtain an average overlap rate; The specific formula for calculating the average overlap rate is as follows: （9） in, Represents the overlapping area between target candidate boxes, represents the area of ​​the target candidate box, Indicates the total number of target candidate boxes.

4. The method for detecting and identifying cervical liquid-based cells based on deep learning according to claim 2, characterized in that: The abnormal cell labels specifically include: low-grade squamous intraepithelial lesion cells, atypical squamous epithelial cells of undetermined significance, high-grade squamous intraepithelial lesion, atypical glandular cells, uterine adenocarcinoma cells, and clue cells.

5. A deep learning-based cervical liquid-based cell detection and identification system, the system being used to implement the deep learning-based cervical liquid-based cell detection and identification method according to any one of claims 1 to 4, characterized in that: Including medical image cropping module, image annotation module, data division module, image enhancement module, network training module, and target detection module; Medical image cropping module: crops the collected cervical liquid-based cell images; Image annotation module: annotates image data and uploads the image to the system until all image data are annotated; Data partitioning module: divides the image dataset into training set, validation set, and test set randomly, and the partition ratio can be customized; Image enhancement module: performs data enhancement on the dataset, including but not limited to inscribed circle rotation image enhancement, image quartering enhancement, Gaussian filtering, image rotation, and grayscale histogram equalization; Network training module: inputting the data-augmented image data into the improved YOLOv5 network model for model training, wherein the upsampling method of the feature fusion module of the improved YOLOv5 network model adopts an improved bilinear interpolation method; Target detection module: batch detects images to be tested, and displays the number of abnormal cell categories in all test images through the system display page and automatically performs statistics.

Images