Intelligent pathological image diagnosis system based on cloud service

Through an intelligent pathological image diagnosis system based on cloud services, the detection of abnormal cells and pathological grading are used to use deep learning algorithms to solve the shortcomings of automated diagnosis in the existing technology, and achieve high accuracy and large-scale promotion of early cancer diagnosis.

CN113130049BActive Publication Date: 2024-11-26HEILONGJIANG JIZHITONG INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110406946.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-04-15
Publication Date
2024-11-26
Estimated Expiration
2041-04-15

AI Technical Summary

Technical Problem

The existing automated pathological image diagnosis technology is not yet mature in the detection of abnormal tissues or abnormal cells and pathological hierarchical diagnosis, and cannot achieve large-scale promotion. Due to weak hardware and shortage of doctors, it is difficult to achieve effective diagnosis of early cancer.

Method used

Develop an intelligent pathological image diagnosis system based on cloud services, including information acquisition module, pathological slice image preprocessing module, pathological slice image intelligent diagnosis module and report generation module, and use object detection model and deep neural network for abnormal cell detection and lesion degree grading , combine multiple deep learning algorithms to achieve automated diagnosis.

Benefits of technology

Automatic pathological diagnosis and auxiliary diagnosis are realized, the detection accuracy of abnormal tissues and abnormal cells and the accuracy of pathological diagnosis grading are improved, the burden on doctors is reduced, and large-scale promotion and effective diagnosis of early cancer are supported.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113130049B_ABST
    Figure CN113130049B_ABST
Patent Text Reader

Abstract

An intelligent pathological image diagnosis system based on cloud services, the present invention relates to artificial intelligence, medical diagnosis, deep learning, and auxiliary diagnosis systems. The present invention is to solve the problem that the prior art does not have an intelligent pathological image diagnosis system based on cloud services. The system includes: an information acquisition module for information entry personnel to sequentially enter patient basic information and slice information in the cloud diagnosis system; a pathological slice image preprocessing module for acquiring and processing slice images to obtain slice panoramas and slice tile images; a pathological slice image intelligent diagnosis module for outputting digital pathology files; a preliminary diagnosis module for generating preliminary diagnosis opinions and preliminary analysis reports based on digital pathology files; a report generation module for doctors to view preliminary diagnosis opinions and preliminary analysis reports, conduct manual review and diagnosis, and generate a final analysis report. The present invention is used in the field of intelligent pathological image diagnosis based on cloud services.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to artificial intelligence, medical diagnosis, deep learning, and auxiliary diagnosis systems. Background Art

[0002] Cancer is the number one killer that threatens people's lives and has become a social problem in my country. Limited by my country's economic level and detection level, people generally lack awareness of cancer prevention, and the actual situation may be even more pessimistic. The long incubation period of cancer and the difficulty of early detection due to asymptomatic disease, which leads to the missing of the best treatment time, are the fundamental reasons for the high cancer mortality rate. Early detection and early treatment are the most effective means to deal with cancer. Pathological diagnosis is known as the "gold standard" for disease diagnosis and is currently the most effective means of diagnosis. Traditional pathological diagnosis first takes human samples (tissues or cells) and prepares slides, and then relies on doctors to "manually operate and observe with the naked eye" for manual reading. There are two major pain points: (1) The hardware of the pathology department of grassroots hospitals is weak and the means are backward. The accuracy of cancer diagnosis is generally low, and there are a large number of misdiagnoses and missed diagnoses. (2) Due to the large workload of pathological diagnosis, the long training cycle of high-level pathologists, and the serious shortage of them, it is impossible to achieve routine cancer screening. In recent years, artificial intelligence and big data technologies have been successfully used in pathology, and automatic pathology reading technology has emerged. Currently, the equipment on the market has basically completed the automated scanning of pathology samples, and supports digitization and Internet sharing, quality control, archiving and remote diagnosis of pathology data, but it is still immature in auxiliary diagnosis or automatic diagnosis and cannot be promoted on a large scale.

[0003] There are still two key issues that need to be urgently resolved in automatic film reading technology: one is the detection of abnormal tissue or abnormal cells. Pathological images are rich in content, complex in situation, and have a large number of cells. It is very difficult to accurately find abnormal tissue or abnormal cells. The second is pathological grading diagnosis. It is an urgent need for pathologists to determine the degree of cancer based on the number of abnormal cells, the level of lesions, the infiltration, atypia, and proliferation activity of abnormal tissues. The pathological grading diagnosis function can not only provide a basis for doctors to formulate treatment plans, but also evaluate the effectiveness of treatment plans. Summary of the invention

[0004] The present invention aims to solve the problem that the prior art does not have an intelligent pathology image diagnosis system based on cloud services.

[0005] The cloud-based intelligent pathology image diagnosis system includes:

[0006] Information acquisition module, pathological slice image preprocessing module, pathological slice image intelligent diagnosis module, preliminary diagnosis module, report generation module;

[0007] The information collection module is used by information input personnel to input basic patient information and slice information in the cloud diagnosis system in sequence;

[0008] The pathological slice image preprocessing module is used to acquire and process slice images to obtain slice panorama images and slice tile images;

[0009] The pathological slice image intelligent diagnosis module is used to perform preliminary diagnosis and annotation on the slice tile image and output a digital pathology file;

[0010] The preliminary diagnosis module is used to generate a preliminary diagnosis opinion and a preliminary analysis report based on the digital pathology file;

[0011] The report generation module is used by doctors to review preliminary diagnosis opinions and preliminary analysis reports, conduct manual review and diagnosis, and generate final analysis reports.

[0012] Furthermore, the information collection module is used by the information input personnel to input the basic information of the patient and the slice information in the cloud diagnosis system in sequence; specifically:

[0013] The basic information of the patient includes: the patient's age, genetic disease history, menstrual history, and previous treatment drugs;

[0014] The slice information includes: slice number, slice type, staining method, and sampling time.

[0015] Furthermore, the pathological slice image preprocessing module is used to acquire and process slice images to obtain slice panorama images and slice tile images; the implementation steps are as follows:

[0016] (1) Obtaining slice scan images: Slice scan images are obtained from:

[0017] 1) The microscope scanning client moves to take the slice pathology image in the current field of view, obtains the slice scan image, and transmits it to the cloud diagnosis system in real time;

[0018] 2) The cloud diagnosis system supports uploading slice scans of different formats scanned by other scanning terminals on the web page;

[0019] (2) stitching the slice scans into a panoramic image;

[0020] The sliced ​​panorama is processed using a pyramid cutting method to obtain M×N non-overlapping small images, and the position correspondence between the non-overlapping small images and the panorama is recorded to obtain a sliced ​​tile image.

[0021] The M and N represent the length and width of the non-overlapping small images respectively.

[0022] Furthermore, the pathological slice image intelligent diagnosis module performs preliminary diagnosis and annotation on the slice tile image and outputs a digital pathology file; the specific process is:

[0023] (1) Determine the target detection model of the intelligent diagnosis module of the pathological slice image, use the target detection model to detect the slice tile image, and obtain abnormal cells;

[0024] (2) grading the degree of cytopathic effect of abnormal cells to obtain the abnormal grade of abnormal cells;

[0025] (3) Score the disease level of abnormal cells to obtain the patient's disease level;

[0026] (4) A preliminary diagnosis and annotation are made based on the abnormal level of abnormal cells and the patient's condition to obtain a digital pathology file.

[0027] Furthermore, the preliminary diagnosis module is used to generate a preliminary diagnosis opinion and a preliminary analysis report based on the digital pathology file; the implementation steps are as follows:

[0028] Based on the digital pathology files, a preliminary diagnosis was obtained;

[0029] Generate a preliminary analysis report based on the preliminary diagnosis, basic patient information, and slice information;

[0030] The basic information of the patient includes: the patient's age, genetic disease history, menstrual history, and previous treatment drugs;

[0031] The slice information includes: slice number, slice type, staining method, and sampling time.

[0032] Furthermore, the report generation module is used by doctors to review preliminary diagnosis opinions and preliminary analysis reports, conduct manual review and diagnosis, and generate final analysis reports; the implementation steps are as follows:

[0033] The doctor checks the digital pathology file and preliminary analysis report, gives the final diagnosis conclusion, and generates a diagnosis report through the report generation module.

[0034] Furthermore, the determination process of the target detection model of the pathological slice image intelligent diagnosis module is as follows:

[0035] 1) Select samples containing abnormal cells as experimental samples and enter them into the experimental sample database;

[0036] 2) Label all experimental samples in the experimental sample database; specifically:

[0037] A semantic segmentation model is used to segment all abnormal cells in the experimental sample database;

[0038] Use a deep neural network model to classify the segmented cells;

[0039] 3) The experimenter checks the annotation results in 2), corrects the wrong annotations, and completes the missing annotations to obtain the verified annotated experimental samples;

[0040] 4) Input the verified labeled experimental samples into different target detection models for training to obtain different trained target detection models;

[0041] Calculate the accuracy, precision, recall, F1-Measure, and loss value of inputting the verified labeled experimental samples into different trained target detection models;

[0042] The target detection model with the highest accuracy is selected as the target detection model of the intelligent diagnosis module of pathological slice images.

[0043] Furthermore, in (2), the abnormal cells are graded for the degree of cytopathic effect to obtain the abnormal grade of the abnormal cells; the specific process is:

[0044] Step 1: Obtain cell diagnostic indicators;

[0045] Step 2: formulating the obtained cell diagnostic indicators;

[0046] Step 3: Classify the diagnostic index levels according to the extracted diagnostic indexes;

[0047] Step 4: Establish multiple sets of training data sets according to the classification levels;

[0048] Step 5: Learn the numerical range of the diagnostic indicator level and determine the score of each divided level;

[0049] Step 6: Fitting the relationship between each level of abnormal cells and diagnostic indicators;

[0050] Step 7: Calculate the probability of abnormal cells at each lesion level and determine the lesion level of the cell.

[0051] Furthermore, the (3) steps of scoring the abnormal cells to obtain the patient's disease level are as follows:

[0052] The cell diagnostic index acquisition module acquires cell diagnostic indexes according to TBS diagnostic criteria;

[0053] The cell lesion grading module is based on various diagnostic indicators of abnormal cells and uses the mixed Gaussian model M corresponding to each diagnostic indicator. i Determine the degree level of different diagnostic indicators of cells and the corresponding probability value x i , and use the relationship model between diagnostic indicators and abnormal cell lesion levels to obtain the predicted values ​​of abnormal cells at various lesion levels, and take the lesion level with the largest predicted value as the lesion level of the cell;

[0054] A cell microbial infection detection module determines whether the cell image is infected with microorganisms and determines the type of microbial infection;

[0055] The abnormal cell lesion level adjustment module adjusts the weight vector ω of the relationship model between the diagnostic index and the abnormal cell lesion level according to the patient's age, menstrual period, radiotherapy history information and microbial infection information. j Adjust the weights of diagnostic indicators in

[0056] The patient cancer level determination module determines the patient cancer level according to the lesion level of all cells adjusted by the abnormal cell lesion level adjustment module.

[0057] The beneficial effects of the present invention are:

[0058] The present invention proposes an intelligent pathological image diagnosis system based on cloud services. The overall technical solution is as follows: Figure 1 As shown in the figure, the hardware part of the system consists of a microscopic imaging module, a 3D PTZ control module, an industrial-grade image acquisition module, a high-performance server and a general server cluster, and the software part consists of a pathological slice image acquisition module, a pathological slice image preprocessing module, a pathological slice image intelligent diagnosis module, a preliminary diagnosis module, and a report generation module. The system collects and processes patient information and pathological slice images, makes a preliminary diagnosis and annotation of pathological slice images, generates preliminary diagnosis opinions and preliminary analysis reports, generates final analysis reports after review by doctors, and continuously optimizes the deep learning model used in diagnosis. It performs outstandingly in automatically detecting abnormal tissues and cells and realizing pathological diagnosis grading, and realizes automatic pathological diagnosis and auxiliary pathological diagnosis.

[0059] The intelligent pathological image diagnosis system of the cloud service of the present invention aims at automatic pathological diagnosis and auxiliary pathological diagnosis, automatically detects abnormal tissues and abnormal cells and realizes pathological diagnosis grading, develops multiple software modules, and adopts multiple deep learning algorithms. Based on this, a complete set of intelligent pathological image diagnosis system of the cloud service is established by combining software and hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a model framework diagram of the present invention. DETAILED DESCRIPTION

[0061] Specific implementation method 1: This implementation method is an intelligent pathological image diagnosis system based on cloud services, which is characterized by comprising:

[0062] Hardware: HD digital camera, digital camera adapter, electric stage, joystick, OLYMPUS optical microscope, microscope light source box, control box, 4 GPU servers; Software: information acquisition module, pathological slice image preprocessing module, pathological slice image intelligent diagnosis module, preliminary diagnosis module, report generation module;

[0063] The information collection module is used by information input personnel to sequentially input basic patient information and cervical slice information in the cloud diagnosis system;

[0064] The pathological slice image preprocessing module is used to acquire and process the cervical slice image to obtain a cervical slice panorama image and a cervical slice tile image;

[0065] The pathological slice image intelligent diagnosis module is used to perform preliminary diagnosis and annotation on the cervical slice tile image and output a digital pathology file;

[0066] The preliminary diagnosis module is used to generate a preliminary diagnosis opinion and a preliminary analysis report based on the digital pathology file;

[0067] The report generation module is used by doctors to review preliminary diagnosis opinions and preliminary analysis reports, conduct manual review and diagnosis, and generate final analysis reports.

[0068] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the information collection module is used by the information input personnel to input the basic information of the patient and the cervical slice information in the cloud diagnosis system in sequence; specifically:

[0069] The basic information of the patient includes: the patient's age, genetic disease history, menstrual history, and previous treatment drugs;

[0070] The cervical section information includes: section number, cervical section type, staining method, and sampling time.

[0071] The other steps and parameters are the same as those in the first embodiment.

[0072] Specific implementation method three: This implementation method is different from specific implementation method one or two in that the pathological slice image preprocessing module is used to obtain and process cervical slice images to obtain cervical slice panorama images and cervical slice tile images; the implementation steps are as follows:

[0073] (1) Obtaining cervical biopsy scans: The sources of cervical biopsy scans include:

[0074] 1) The microscope scanning client moves to take the pathological image of the cervical section in the current field of view, obtains the cervical section scan image, and transmits it to the cloud diagnosis system in real time;

[0075] 2) The cloud diagnosis system supports uploading cervical slice scans in different formats scanned by other scanning terminals on the web page, such as svs, jp2, lif, vsi, tif, tiff, etc.;

[0076] The microscope scanning client is composed of a microscopic imaging system, a three-dimensional control pan-tilt head, and an industrial-grade image acquisition module. It uses a variable-step hill climbing method to achieve focus search and a low gray value statistical method to evaluate image clarity, ensuring that the best local field of view is obtained.

[0077] (2) splicing the cervical slice scans into a panoramic image and unifying the image format of the cervical slice panoramic image;

[0078] The cervical slice panorama is processed using the pyramid cutting method to be processed into M*N non-overlapping small images, and the position correspondence between the non-overlapping small images and the panorama is recorded to realize the cell positioning function, and the cervical slice tile image is obtained, that is, a number of folders storing cervical slice images of different numbers and the same range. The folders are arranged in order from low to high according to the internal image resolution, representing different zoom levels, which are used for zoom display of the panorama;

[0079] The M and N represent the length and width of the non-overlapping small images respectively.

[0080] Process the panorama into 1024*1024 non-overlapping small images for use in subsequent model processing, and record the position correspondence between the non-overlapping small images and the panorama.

[0081] The other steps and parameters are the same as those in the first or second embodiment.

[0082] Specific implementation method 4: This implementation method is different from any one of specific implementation methods 1 to 3 in that the pathological slice image intelligent diagnosis module performs preliminary diagnosis and annotation on the cervical slice tile image and outputs a digital pathology file; the specific process is:

[0083] (1) Determine the target detection model of the intelligent diagnosis module of pathological slice images, and use the target detection model to detect cervical slice tiles to obtain abnormal cells; the network adopts a one-stage method to improve the real-time detection speed of a large number of images; the input end uses Mosaic data enhancement to improve the detection effect of small targets, which is conducive to improving the detection rate of cells;

[0084] (2) grading the degree of cytopathic effect of abnormal cells to obtain the abnormal grade of abnormal cells;

[0085] According to the method described in the patent "Abnormal Cell Grading System", the abnormal grade of abnormal cells is obtained;

[0086] (Information such as the number, size, color, texture, etc. of abnormal cells or tissues);

[0087] (3) Score the disease level of abnormal cells to obtain the patient's disease level; the whole process is:

[0088] According to the method described in the patent "A cancer degree grading system", the patient's condition level is obtained;

[0089] (4) A preliminary diagnosis and annotation are made based on the abnormal level of abnormal cells and the patient's condition to obtain a digital pathology file.

[0090] Use target detection models to detect abnormal cells and tissues;

[0091] The nucleus and cytoplasm were segmented using the segmentation model;

[0092] Calculate and analyze common parameters of the cell nucleus, cytoplasm, and tissue to obtain information such as the number, size, color, and texture of abnormal cells or tissues;

[0093] According to the pathological diagnostic criteria, the lesion level of abnormal cells and abnormal tissues is obtained, and combined with the basic information of the patient, the degree of the patient's lesion is obtained;

[0094] The lesion level and annotation information of abnormal cells and tissues, and the patient's disease level score are integrated to obtain a digital pathology file.

[0095] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0096] Specific implementation method 5: This implementation method is different from the specific implementation methods 1 to 4 in that the preliminary diagnosis module is used to generate a preliminary diagnosis opinion and a preliminary analysis report based on the digital pathology file; the implementation steps are as follows:

[0097] Based on the digital pathology files, get a preliminary diagnosis, including: bacterial infection, diagnosis results, and doctor's advice;

[0098] Generate a preliminary analysis report based on the preliminary diagnosis, basic patient information, and cervical biopsy information;

[0099] The basic information of the patient includes: the patient's age, genetic disease history, menstrual history, and previous treatment drugs;

[0100] The cervical section information includes: section number, cervical section type, staining method, and sampling time.

[0101] Based on the digital pathology files, a preliminary diagnosis was obtained, such as using "sample satisfaction: satisfied, cell count: >5000, inflammatory cell count: none" to describe the basic sample information, "suggestive of bacterial infection, suggestive of actinomycete infection, suggestive of fungal infection, endocervical cells, metaplastic cells, similar to cytological changes caused by herpes virus" to describe the bacterial infection, "preliminary diagnosis of high-grade squamous intraepithelial lesion, but combined with the patient's age and previous treatment drugs, it was finally determined to be low-grade squamous intraepithelial lesion, positive staining cells" to describe the diagnosis result, and "regular review, 4-6 month review, clinical colposcopy, biopsy when necessary, please combine clinical practice, if the number of skin cells does not meet the detection standards, it is recommended to re-sample" to describe the doctor's advice; a preliminary analysis report was generated based on the preliminary diagnosis, basic patient information, and cervical biopsy sample information.

[0102] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0103] Specific implementation method 6: This implementation method is different from the specific implementation methods 1 to 5 in that the report generation module is used by doctors to view preliminary diagnosis opinions and preliminary analysis reports, conduct manual review and diagnosis, and generate final analysis reports; the implementation steps are as follows:

[0104] The doctor checks the digital pathology file and preliminary analysis report, gives the final diagnosis conclusion, and generates a diagnosis report through the report generation module.

[0105] The other steps and parameters are the same as those in Specific Implementations 1 to 5.

[0106] Specific implementation method 7: This implementation method is different from any one of specific implementation methods 1 to 6 in that the target detection model of the pathological slice image intelligent diagnosis module is determined as follows:

[0107] 1) Select samples containing abnormal cells from the three experimental samples of the cloud diagnosis system, the server-specified folder, and the uploaded panoramic images in other formats as experimental samples, and enter them into the experimental sample database;

[0108] 2) Label all experimental samples in the experimental sample database; specifically:

[0109] The UNet network is used to segment all abnormal cells in the experimental sample database;

[0110] Use the Resnet model to classify the segmented cells;

[0111] Cell segmentation: For all cells in the experimental sample, the segmentation model is first used to segment the cell nucleus, and then the GAN network is used to repair the incomplete cytoplasm.

[0112] Classification of abnormal tissues and abnormal cells; Classification models are used to classify abnormal tissues and abnormal cells.

[0113] The UNet network includes a downsampling module and two upsampling modules;

[0114] The downsampling module reduces the image size and extracts superficial features through convolution and pooling operations, while the upsampling module mainly restores the image size and extracts deep features through deconvolution and convolution operations. This model design enables the feature extraction module to focus on the features of the cell nucleus and cytoplasm in the image at the same time, while the separate upsampling network allows the segmentation network to focus on its own segmentation tasks, thereby improving the segmentation effect.

[0115] The Resnet model is used to classify abnormal cells. The network adds residual units through a short-circuit mechanism, which avoids the degradation problem of deep networks while maintaining the complexity of the network, making the abnormal cell classification task more effective.

[0116] 3) The experimenter checks the annotation results in 2), corrects the wrong annotations, and completes the missing annotations to obtain the verified annotated experimental samples;

[0117] 4) Input the verified labeled experimental samples into the target detection model for pre-training to obtain a pre-trained target detection model;

[0118] The target detection models are Yolo5 model and Faster R-CNN model;

[0119] The Yolo5 model uses the pytorch deep learning framework, and the Yolo5 model file format is pth;

[0120] The effect verification code is specified on the server side to facilitate unified evaluation of model effects, without the need for users to upload it separately.

[0121] Deep learning framework: support mainstream deep learning frameworks such as pytorch, caffe, tensorflow, keras, etc.

[0122] Experimental model: supports common deep learning model types such as pt, pth, pkl, h5, ckpt, etc.

[0123] The training code should contain functions with the same names: model_train, model_test, model_save, train_loss, val_loss.

[0124] Calculate the accuracy, precision, recall, F1-Measure, loss value, etc. of the verified labeled experimental samples input into the pre-trained Yolo5 model;

[0125] Calculate the accuracy, precision, recall, F1-Measure, loss value, etc. of the verified labeled experimental samples input into the pre-trained Faster R-CNN model;

[0126] The results are shown in Table 1;

[0127] When different experimental tasks use the same model, the model experimental information of the model in different experiments is obtained, and weighted average is performed to obtain the comprehensive experimental information of the model; when an experimental task uses different models, the experimental information of each model in the experimental task is counted separately, and the experimental results can be compared in the experimental tasks.

[0128] 5) The experimenter checks the results of 4) and expands the experimental sample data set; the specific process is:

[0129] The experimenters checked the experimental results of Yolo5 and Faster R-CNN, and based on the doctors' suggestions, selected 458 typical abnormal tissues or cells from the correctly detected samples to expand the experimental sample data set, and selected 109 mispredicted tissues or cells that caused interference from the misdetected samples to establish a separate data set;

[0130] 6) Input the expanded experimental sample data set into the pre-trained Yolo5 model to obtain the trained Yolo5 model and calculate the accuracy;

[0131] Input the expanded experimental sample data set into the pre-trained Faster R-CNN model to obtain the trained Faster R-CNN model and calculate the accuracy;

[0132] Select the model with the highest accuracy as the target detection model for the intelligent diagnosis module of pathological slice images;

[0133] Use incremental learning to learn the features of selected typical abnormal tissues or cells and expand the parameter scale of Yolo5 and Faster R-CNN models;

[0134] Use online learning to build a Resnet34 model to learn the features of incorrectly predicted tissues or cells to eliminate interference factors in the classification results;

[0135] Compare the accuracy of the optimized Yolo5 model, Faster R-CNN model, and the target detection model currently being used by the intelligent diagnosis task, and select the Yolo5 model with the highest accuracy to replace the model used by the intelligent diagnosis task.

[0136] Model optimization, select the best model to replace the model used in the intelligent diagnosis task. The implementation steps are as follows:

[0137] S130 supports multiple model optimization methods: incremental learning, online learning, and offline training. The optimization steps of different methods are as follows:

[0138] Incremental learning: Use the initial training set to preliminarily train the model. Each time the training set is updated, the model is trained again using the updated data. This allows the model to retain the original training parameters while expanding the parameter scale, thereby achieving the goal of retaining the original experience and adding new knowledge, thereby achieving the effect of continuously optimizing the model parameters.

[0139] Online learning: For typical abnormal tissues or cells in the data set, train them one by one. When each piece of data reaches the effect of making the model error rate value very small, the next piece of abnormal tissue or cell data is automatically used; for incorrectly predicted tissues or cells, a new model is trained in the same way;

[0140] Offline training: R&D personnel improve the network structure of the existing model, or design a new network structure, use the same dataset for model training, upload the model, and update the list of optional models for the experiment;

[0141] S131. Compare the accuracy of the optimized model with the model used for the intelligent diagnosis task, and obtain the best model for use in the intelligent diagnosis task.

[0142] Input the verified labeled experimental samples into the target detection model for pre-training to obtain a pre-trained target detection model;

[0143] The implementation steps are as follows:

[0144] Experimental requirements: Compare the accuracy of different target detection models;

[0145] Sample data set: 30 clinical trial data samples, including four abnormal cell grades: ASC-US, ASC-H, LSIL, and HSIL.

[0146] Experimental tasks: ① Select the one-stage target detection model Yolo5 ② Select the two-stage target detection model FasterR-CNN.

[0147] Select experimental samples to establish a sample data set and select a suitable model for the experiment. The implementation steps are as follows:

[0148] S100. Establish experimental tasks according to experimental requirements, complete resource allocation through the experimental task parallel processing module, and realize efficient parallelization between tasks;

[0149] S101, select appropriate experimental samples and experimental models for each experimental task in turn;

[0150] S102: If there is no suitable model, support online model training. The specific process is as follows: upload the training code, use the pre-trained model to initialize the model parameters; set the number of training rounds, termination conditions and other constraint information, and train the model online. There are two types of pre-trained models: one is the online public pre-trained model; the other is the pre-trained model trained internally in the cloud diagnosis system.

[0151] S103. Conduct experiments based on S101 and S102.

[0152] Table 1. Statistical comparison of experimental information of Yolo5 model and Faster R-CNN model

[0153] TP FP FN Recall % Accuracy % Yolo5 987 56 238 80.57 94.63 Faster R-CNN 951 105 122 88.63 90.06

[0154] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0155] Specific implementation example eight: This implementation example is different from specific implementation examples one to seven in that in (2), abnormal cells are graded for cytopathic degree to obtain the abnormal grade of abnormal cells; the specific process is as follows:

[0156] Step 1: Obtain cell diagnostic indicators;

[0157] Step 2: formulating the obtained cell diagnostic indicators;

[0158] Step 3: Classify the diagnostic index levels according to the extracted diagnostic indexes;

[0159] Step 4: Establish multiple sets of training data sets according to the classification levels;

[0160] Step 5: Learn the numerical range of the diagnostic indicator level and determine the score of each divided level;

[0161] Step 6: Fitting the relationship between each level of abnormal cells and diagnostic indicators;

[0162] Step 7: Calculate the probability of abnormal cells at each lesion level and determine the lesion level of the cell.

[0163] Step 1: Extract diagnostic indicators: Extract diagnostic indicators of cells from cytological diagnostic standards; the specific process is:

[0164] According to the cytological diagnostic criteria, 18 representative and targeted diagnostic indicators were obtained by summarizing the cell's morphological characteristics, color characteristics, texture characteristics, and cell cluster characteristics.

[0165] Among them, the morphological characteristic indicators include nuclear-cytoplasmic ratio, nuclear division degree, nuclear polarity, nuclear eccentricity, nuclear atypia, cell circular fit, and cell nuclear area coefficient; the color characteristic indicators include vacuoles / cytoplasm area ratio, nucleolus / nucleus area ratio, and keratinization degree; the texture characteristics include the concave area of ​​nuclear grooves, nuclear staining depth, nuclear staining uniformity, cytoplasm richness, and cell outline clarity; the cell cluster characteristics include glandular cell disorder, cell cluster crowding, and cell cluster size distribution;

[0166] Step 2: Define the indicator formula: Use the cell morphology priors and basic mathematical formulas to define the calculation formula of the diagnostic indicator; that is, formulate the diagnostic indicator in step 1; the specific process is:

[0167] In the shape feature, the specific formula of the nucleus-to-cytoplasm ratio is:

[0168]

[0169] Among them, R is the nucleus-cytoplasm ratio, A c is the cell area, A n is the cell nucleus area;

[0170] The calculation method of nuclear division degree requires first calculating the curvature of all points on the contour, and selecting the cumulative sum of the 10 points with the largest curvature as the nuclear division degree. The calculation formula is as follows:

[0171]

[0172] Among them, x p 、x pp represent the first and second derivatives of the x component of the curve, y p ,y pp They represent the first and second derivatives of the y component of the curve respectively;

[0173] The specific formula for nuclear polarity is:

[0174]

[0175] Among them, P n is the nuclear polarity value, N is the total number of cells in the slide, θ i is the eccentricity angle of the i-th cell, is the average eccentricity angle, x1 is the horizontal coordinate of the center of the cell, y1 is the vertical coordinate of the center of the cell, x2 is the horizontal coordinate of the center of the cell nucleus, and y2 is the vertical coordinate of the center of the cell nucleus;

[0176] The specific formula for nuclear eccentricity is:

[0177]

[0178] Among them, d is the nuclear eccentricity, P1(x1,y1) is the cell center, and P2(x2,y2) is the cell nuclear center;

[0179] The specific formula for nuclear atypia is:

[0180]

[0181] Where C is the degree of nuclear atypia, D is the short axis of the rectangle circumscribing the nucleus, L is the long axis of the rectangle circumscribing the nucleus, and A n is the cell nucleus area;

[0182] The specific formula for the cell circular fit is:

[0183]

[0184] Among them, A n is the area of ​​the cell nucleus, d1 and d2 are the major axis and minor axis of the cell nucleus respectively, and P is the circumference of the cell nucleus. The calculation process is as follows:

[0185] For points on the boundary of the cell nucleus, the Euclidean distance between two adjacent points is calculated in turn, and finally the sum of the Euclidean distances of all adjacent points on the boundary is obtained, which is the perimeter of the boundary of the cell nucleus; the Euclidean distance formula between two points is shown as follows:

[0186]

[0187] Among them, (x i ,y i )、(x i+1 ,y i+1 ) are two adjacent points on the boundary of the cell nucleus;

[0188] The specific formula for the cell nuclear area coefficient is:

[0189]

[0190] Among them, A index is the cell nuclear area coefficient, A n is the cell nucleus area, A m is the mean area of ​​the nuclei of cells in the same layer;

[0191] In the color features, the HSV color model is mainly used, which comprehensively considers the hue, saturation, brightness information, and the vacuolar / cytoplasmic area ratio K vacuoles The specific formula is:

[0192]

[0193] Among them, HSV vacuoles For whiteness, A vacuoles is the cavitation area, A cyis the cytoplasm area;

[0194] Nucleolus / nucleus area ratio K nucleolus The specific formula is:

[0195]

[0196] Among them, HSV nucleolus A is the degree of blackness, nucleolus is the nucleolus area output by the model, A n is the cell nucleus area;

[0197] The calculation formula of angularity is:

[0198]

[0199] Among them, K orange is the keratinization degree, HSV orange For orange, A orange is the keratinized area;

[0200] Among the texture features, the concave area A in the nuclear groove groove The specific formula is:

[0201]

[0202] The Line_gauss algorithm is used to detect all lines in the cell nucleus and add up their widths. n represents the number of lines detected, and x represents the sum of all lines in the nucleus. i Indicates the width of each line;

[0203] The specific formula for the depth of nuclear staining is:

[0204]

[0205] in, represents the depth of nuclear staining, f(x,y) is the gray value of the position (x,y) of the nuclear image, l is the length of the nuclear image, and d is the width of the nuclear image;

[0206] The specific formula for nuclear staining uniformity is:

[0207]

[0208] Where S represents the unevenness of nuclear staining, f(x,y) is the gray value of the nuclear image at position (x,y), l is the length of the nuclear image, and d is the width of the nuclear image;

[0209] The specific formula for cytoplasm richness is:

[0210]

[0211] Among them, T cy represents the richness of cytoplasm, S(x,y) is the gradient value of the Sobel convolution kernel at the point (x,y) in the cytoplasm of the image, and n is the number of cytoplasm pixels;

[0212] The specific formula for cell outline clarity is:

[0213]

[0214] Among them, F represents the clarity of the cell contour, G(x,y) is the gradient value after the Laplace operator is convolved with the image at the (x,y) point in the cytoplasm nuclear contour;

[0215] In the cell cluster characteristics, the specific formula for the disorder of glandular cells is:

[0216]

[0217] Among them, H gc represents the disorder degree of glandular cells, p(x i ) is the probability of glandular cells in the unit rectangle, N g is the number of unit rectangles;

[0218] The specific formula for cell cluster crowding is:

[0219]

[0220] Among them, O cm is the cell cluster crowding degree, A overlap is the overlapping cell area, A ncm is the total area of ​​cell nuclei in the cell cluster, N ncm is the number of overlapping cells;

[0221] The specific formula for the cell cluster size distribution is:

[0222]

[0223] Among them, B cm Indicates the size distribution of cell clusters, A i is the area of ​​the ith cell cluster, is the average area of ​​cell clusters, N cm is the number of cell clusters;

[0224] Step 3: Classify the indicators: Preliminary classification of the extracted diagnostic indicators; the classification results are as follows:

[0225] Nuclear-cytoplasmic ratio: normal, slightly increased, increased, high, severely increased;

[0226] Mitotic rate: normal period, mitotic period, multinucleate period;

[0227] Nuclear polarity: obvious, vague, disappeared;

[0228] Nuclear eccentricity: no deviation, deviation within normal range, severe deviation;

[0229] Nuclear atypia: no atypia, mild atypia, severe atypia;

[0230] Cell circular fit: low fit, basic fit, high fit;

[0231] Nuclear area coefficient: 1-1.5, 1.5-2.5, 2.5-3, >3;

[0232] Vacuole / cytoplasm area ratio: with vacuoles, without vacuoles;

[0233] Nucleolus / nucleus area ratio: normal nucleolus, small nucleolus, large nucleolus, no nucleolus;

[0234] Degree of keratinization: no keratinization, slight keratinization, severe keratinization;

[0235] Concave area of ​​nuclear groove: with nuclear groove, without nuclear groove;

[0236] Nuclear staining depth: dark staining, normal staining, light staining;

[0237] Nuclear staining uniformity: uniform staining, uneven staining;

[0238] Cytoplasm richness: thick, mature, diverse, and scanty;

[0239] Cell outline clarity: clear, fuzzy;

[0240] Glandular cell disorder: high, medium, low;

[0241] The degree of cell cluster crowding: crowded, normal, sparse;

[0242] Distribution of cell cluster sizes: different sizes, consistent sizes;

[0243] The fourth step is to establish multiple sets of training data sets according to the classification levels; the specific process is as follows:

[0244] Step 41: Obtain cell slices;

[0245] Step 42: Obtain cell images from cell slices, segment the images using the Unet model, obtain all cell images in the images, and record the contour positions of the cell nucleus and cytoplasm in the cell images; the cells include abnormal cells and normal cells;

[0246] Step 43: Use the preliminary classification results of diagnostic indicators as the label of cells, and label all diagnostic indicators of cells in sequence. The labeling format is such as "nuclear-cytoplasmic ratio-normal, nuclear division-mitosis period, nuclear polarity-obvious". After the labeling is completed, 18 groups of labels will be obtained. Each group of labels is used as the division standard of the data set to obtain 18 diagnostic indicator data sets;

[0247] Step 44: Complete the cell labeling task, find abnormal cells, and assign a cell abnormality grade label to each abnormal cell according to the grading specifications of the cytological diagnostic standards.

[0248] The specific process is:

[0249] Taking cervical cells as an example, 100 cervical sections (using liquid-based cytology preparation) were obtained. Cervical exfoliated cells were collected from people of different ages and conditions, including all lesion levels of cytology (The Bethesda System, classification and reporting details of vaginal cytology) diagnostic standards;

[0250] A digital scanner was used to obtain cervical cell images, and then the Unet model was used to segment the images to obtain images of all cells in the image. The contour positions of the nucleus and cytoplasm in the cell images were recorded; these cells included 2345 abnormal cells and 5000 normal cells.

[0251] Using the preliminary classification results of diagnostic indicators as the labeling labels of cervical section cells, all diagnostic indicators of cervical section cells are labeled in sequence, with the labeling format such as "nuclear-cytoplasmic ratio-normal, nuclear division-mitotic phase, nuclear polarity-obvious". After the labeling is completed, 18 groups of labels will be obtained. Using each group of labels as the division standard of the data set, 18 diagnostic indicator data sets are obtained;

[0252] Complete the task of labeling 100 cervical section cells, find abnormal cells, and assign a cell abnormality grade label to each abnormal cell according to the grading standards of the cytological diagnosis standard;

[0253] The step 5 includes learning the numerical range of the diagnostic index level and determining the score of each divided level; extracting the corresponding relationship between the diagnostic index and the abnormal cell level according to the cytological diagnostic standard; training the output result of the training index classifier using the cell abnormality level label, and fitting the relationship between the diagnostic index and the cell abnormality level; the specific process is:

[0254] Step 51: Calculate the diagnostic index values ​​of cells in the data set using the formula in step 2 and normalize them;

[0255] Step 52: Use the mixed Gaussian model to model the level of each diagnostic indicator, that is, each Gaussian component represents a different level, and obtain the diagnostic indicator level model Mi i=0,1,2…18, representing 18 diagnostic indicators; using M i Identify the new cell diagnostic index level and obtain the corresponding probability value x i .

[0256] The step six is ​​to fit the relationship between each level of abnormal cells and diagnostic indicators; the specific process is:

[0257] Step 6.1. According to the cytological diagnostic criteria, each abnormal cell lesion level focuses on observing different aspects of cell morphological characteristics, that is, different diagnostic indicators need to be referred to when judging different abnormal cell lesion levels, and the corresponding relationship between diagnostic indicators and abnormal cell grading is extracted, as shown in Table 2; this table is calibrated by experienced doctors and reflects the doctors' prior knowledge. However, when judging a certain abnormal cell lesion level, the weights of different diagnostic indicators are slightly different;

[0258] Table 2 Correspondence between diagnostic indicators and abnormal cell lesion levels

[0259]

[0260]

[0261] Step 6.2: Calculate the weight vector described in the above steps Among them, j represents the abnormal cell lesion grade, Y j is the predicted value of abnormal cell lesion level, and the formula is as follows:

[0262] Y j =ω j T X j

[0263] The elastic regression model is selected and combined with the cell abnormality level label to train the output results of the diagnostic index classifier to fit the relationship between the diagnostic index and the cell abnormality level:

[0264]

[0265] Among them, y j is the true value of the abnormal cell lesion level, which is set to 1; as shown in Table 1, different diagnostic indicators are required for diagnosis of different lesion levels. The diagnostic indicators required for the jth lesion level are combined into a vector X j , X j The probability value of the diagnostic indicator is x i , obtained by step 52; λ1||ω j ||1. They are the two penalty terms of elastic regression respectively; use the data set in step 44 to train the model and obtain the corresponding weights.

[0266] The step 7 calculates the probability of abnormal cells at each pathological level and determines the pathological level of the cells; the specific process is:

[0267] Step 71: Calculate the diagnostic index of the input cell according to the formula defined in step 2, and then use the Gaussian mixture model to predict the diagnostic index level and probability value of the input cell;

[0268] Step 72: query Table 1 according to the level of the cell diagnostic index to determine the suspected lesion level of the cell;

[0269] Step 73: Based on the obtained probability value and the predicted value calculated in step 62, the probability value of the missing diagnostic indicator level is zero; finally, the predicted values ​​are compared, and the one with the largest predicted value is determined as the pathological level of the cell.

[0270] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0271] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that the abnormal cells are scored for the disease level in (3) to obtain the patient's disease level; the specific process is:

[0272] The equipment includes but is not limited to PCs, workstations, mobile terminal devices, special medical equipment for cancer-related analysis, etc.

[0273] A cancer degree grading system described in this embodiment includes:

[0274] 1. Cervical cell diagnostic index acquisition module, which obtains cervical cell diagnostic indexes according to TBS diagnostic standards;

[0275] The cervical cell diagnostic index can be any diagnostic index determined according to the TBS diagnostic standard. In this embodiment, the diagnostic index is determined based on cell morphology, including: nuclear-cytoplasmic ratio, nuclear division degree, nuclear polarity, nuclear eccentricity, nuclear atypia, cell circular fit, cell nuclear area coefficient, vacuoles / cytoplasm area ratio, nucleolus / nucleus area ratio, keratinization, nuclear groove concave area, nuclear staining depth, nuclear staining uniformity, cytoplasm richness, cell outline clarity, cell cluster characteristics include glandular cell disorder, cell cluster crowding, and cell cluster size distribution;

[0276] 2. Cytopathic grading module, based on various diagnostic indicators of abnormal cells, using the mixed Gaussian model M corresponding to each diagnostic indicator i Determine the degree level of different diagnostic indicators of cells and the corresponding probability value x i, and use the relationship model between diagnostic indicators and abnormal cell lesion levels to obtain the predicted values ​​of abnormal cells at various lesion levels, and take the lesion level with the largest predicted value as the lesion level of the cell;

[0277] The relationship model between the diagnostic index and the abnormal cell pathology level can be integrated into the cell pathology grading module, and can also be called by the cell pathology grading module to implement the above functions; the relationship model between the diagnostic index and the abnormal cell pathology level is pre-trained, and the specific training process includes the following steps:

[0278] 2.1. Based on the cervical slice image and TBS diagnostic criteria, obtain cervical cell diagnostic indicators;

[0279] In this embodiment, the cervical section images are derived from 100 cervical sections, and the prepared cells are collected from people of different ages and different conditions;

[0280] Normalize the cervical cell diagnostic indicators;

[0281] 2.2. Classify the degree of each diagnostic indicator:

[0282] Nuclear area coefficient: 1-1.5, 1.5-2.5, 2.5-3, >3

[0283] Nuclear-cytoplasmic ratio: normal, slightly increased, increased, high, severely increased

[0284] Nuclear atypia: no atypia, mild atypia, severe atypia

[0285] Nuclear staining degree: mildly dark, normal, moderately dark, deeply dark

[0286] Nuclear staining uniformity: uniform staining, uneven staining

[0287] Degree of keratinization: no keratinization, slight keratinization, severe keratinization

[0288] Vacuole / cytoplasm area ratio: with vacuoles, without vacuoles

[0289] Nuclear division rate: normal period, mitotic period, multinucleate period

[0290] Nucleolus / nucleus area ratio: no nucleolus, normal nucleolus, small nucleolus, large nucleolus

[0291] Concave area of ​​nuclear groove: with nuclear groove, without nuclear groove

[0292] Nuclear eccentricity: no deviation, deviation within normal range, severe deviation

[0293] Cytoplasm richness: thick, mature, diverse, few

[0294] Cell outline clarity: clear, fuzzy

[0295] Cell circle fit: low fit, basic fit, high fit

[0296] Glandular cell disorder: high, medium, low

[0297] Cell cluster size distribution: different sizes, consistent sizes

[0298] Cell cluster nuclear polarity: obvious, blurred, disappeared

[0299] Cell cluster crowding: crowded, normal, sparse

[0300] 2.3. Establish a data set, where the data set labels include diagnostic index level labels and abnormal cell lesion level labels;

[0301] The abnormal cell lesion grades include: ASC-US, ASC-H, LSIL, HSIL, SCC keratinization, SCC non-keratinization, endocervical adenocarcinoma in situ, endocervical adenocarcinoma, and endometrial adenocarcinoma;

[0302] 2.4. Use the mixed Gaussian model to model the degree of each diagnostic indicator M i , using the Gaussian mixture model M i Learn the probability distribution of diagnostic indicators from the data set and use the Gaussian mixture model M i It can determine the degree of different diagnostic indicators of cells and obtain the probability value x corresponding to the degree level i ;

[0303] 2.5. Constructing a relationship model between diagnostic indicators and abnormal cell pathology levels:

[0304] Y j =ω j T X j

[0305] The elastic regression model is used to fit the relationship between each lesion level of abnormal cells and its corresponding diagnostic index; the optimized objective function is:

[0306]

[0307] Among them, ω j is the weight vector; j represents the lesion level of abnormal cells; Y j is the predicted value of abnormal cell lesion grade; j is the true value of abnormal cell pathology level, which is uniformly set to 1; λ1||ω j ||1. They are the two penalty terms of elastic regression;

[0308] The diagnostic indicators required for the j-th lesion level are composed into a vector X j , X j The probability value of the diagnostic indicator is x i , obtained by step 2.4; the model is trained using the data set of step 2.3 to obtain the relationship between the diagnostic index and the abnormal cell pathology level, that is, the relationship model between the diagnostic index and the abnormal cell pathology level.

[0309] 3. Patient information acquisition module, used to obtain the patient's age, menstrual period, and radiotherapy history information;

[0310] According to the doctor's prior knowledge, the patient's age, menstrual period, and radiotherapy history will change the morphological characteristics of the cells. Therefore, understanding the patient's basic information is helpful to accurately diagnose the lesion grade of abnormal cells. The main information that needs to be understood is: age, whether it is within 10 days from the end of menstruation, and the interval between the last radiotherapy and the cytological sampling examination. Typical effects are: for patients over 55 years old, the cells are mostly pyknotic, that is, the nuclear-cytoplasmic ratio increases; within 10 days from the end of menstruation, the secretion of estrogen is vigorous, and the cells are in the division stage, that is, multinucleation is prone to occur; the interval between the last radiotherapy and the cytological sampling examination is within 2 to 6 months, and the cells degenerate and change, that is, the cytoplasm becomes larger, empty, discolored, deformed, and the nucleus becomes blurred.

[0311] Based on the influence of the above situation, it is necessary to adjust the abnormal cell pathology level through the age influencing factor of the nuclear-cytoplasmic ratio weight, the menstrual influencing factor of the nuclear division weight, and the cell degeneration change recognition model. The age influencing factor of the nuclear-cytoplasmic ratio weight, the menstrual influencing factor of the nuclear division weight, and the cell degeneration change recognition model are as follows:

[0312] a. Prepare the patient age information dataset and calculate the age influencing factor of the nuclear-cytoplasmic ratio weight:

[0313] Prepare 100 negative cervical sections (using liquid-based cytology) from women over 55 years old, use a digital scanner to obtain cervical cell images, then use the Unet model to segment the images, obtain all normal cell images in the images, and record the positions of the cell nucleus and cytoplasm;

[0314] The number of all normal cells n obtained from the age information data set is counted, and then the nuclear-cytoplasmic ratio R, the deviation value σ of the nuclear-cytoplasmic ratio, and the age influence factor λ of the nuclear-cytoplasmic ratio weight of all normal cells are calculated in turn. The specific calculation formula is as follows:

[0315] The formula for calculating the nucleus-to-cytoplasm ratio is:

[0316]

[0317] Among them, A c is the cell area, A nis the cell nucleus area;

[0318] The formula for the deviation of the nucleus-to-mass ratio is:

[0319]

[0320] Among them, R i is the nuclear-cytoplasmic ratio of the current cell, R nor-max is the maximum nuclear-cytoplasmic ratio of normal cells in TBS, the specific value is 2.5, and n is the number of normal cells;

[0321] The calculation formula of age influence factor of nuclear-cytoplasmic ratio weight is:

[0322]

[0323] b. Prepare the patient's menstrual cycle cell dataset and calculate the menstrual influence factor of the nuclear division weight:

[0324] Prepare 100 negative cervical sections within 10 days after the end of menstruation (using liquid-based cytology preparation method), and 100 negative cervical sections beyond 10 days after the end of menstruation (using liquid-based cytology preparation method), and collect cervical exfoliated cells from people of different ages; use a digital scanner to obtain cervical cell images, and then use the Unet model to segment the images to obtain all cell images in the images;

[0325] The total number of cells and the number of cells in the nuclear division phase of each cervical section in the menstrual cycle cell data set were counted respectively, and the mean ratio of the number of nuclear division cells within 10 days after the end of menstruation and 10 days after the end of menstruation to the total number of cells was calculated according to the following formula and

[0326]

[0327]

[0328] Among them, N 10-in-CD (k) is the number of cells in the mitotic phase in one of the cervical sections within 10 days after the end of menstruation, N 10-in-CN (k) is the total number of cells in one of the cervical sections within 10 days after the end of menstruation, N 10-out-CD (k) is the number of cells in the mitotic phase in one of the ectocervical sections taken 10 days after the end of the menstrual period, N 10-out-CN (k) is the total number of cells in one of the ectocervical sections 10 days after the end of menstruation;

[0329] Then calculate the menstrual influence factor η of the nuclear division weight:

[0330]

[0331] c. Prepare a dataset of cell degeneration changes caused by radiotherapy and train a cell degeneration change recognition model:

[0332] Prepare 200 negative cervical sections with radiotherapy-induced cell degeneration (using liquid-based cytology preparation), cervical exfoliated cells were collected from people of different ages, and the interval between radiotherapy and cytology sampling examination ranged from 2 to 24 months;

[0333] Use a digital scanner to obtain cervical cell images, then use the Unet model to segment the images and obtain all radiotherapy cell images in the images. Perform data expansion on the cell images of radiotherapy-induced cell degeneration, and finally obtain 4,000 radiotherapy-induced cell degeneration cells, and assign the cell images a "degeneration change" label;

[0334] The ResNet18 network was used to perform cyclic training on the dataset of cell degeneration caused by radiotherapy, and the model parameters were adjusted to obtain a degeneration cell classifier. This classifier can determine whether the input cells have cell degeneration caused by radiotherapy.

[0335] 4. Cell microbial infection detection module, which determines whether there is microbial infection in the cervical cell image and determines the type of microbial infection; this process is achieved through a neural network model;

[0336] According to the TBS diagnostic criteria, microbial infection can cause a variety of cell morphological changes. Typical cell morphological changes caused by microbial infection include: Trichomonas infection can cause bichromatic cytoplasm changes and worm-like appearance of cytoplasm, that is, the vacuolar / cytoplasmic area ratio increases; Candida infection can cause mild diffuse hyperplasia of squamous epithelium, that is, the nuclear-cytoplasmic ratio increases slightly; herpes virus infection can cause ground-glass nuclear changes, multinucleation, chromatin marginalization, and nuclear plasticity, that is, the nuclear division degree increases, the uniformity of nuclear staining decreases, and the crowding of clustered cells increases; cells infected with cytomegalovirus will characteristically increase in size, the nuclear membrane thickens, and satellite inclusions appear, that is, the nuclear-cytoplasmic ratio increases slightly and the vacuolar / cytoplasmic area ratio increases. Accurately distinguishing the morphological changes caused by microbial infection from those caused by abnormal cell lesions can help to accurately diagnose the lesion grade of abnormal cells.

[0337] The training process of the neural network model for judging whether a cervical cell image has microbial infection and determining the type of microbial infection includes the following steps:

[0338] 4.1. Prepare microbial infected cell dataset:

[0339] Prepare 800 cervical sections with microbial infection (using liquid-based cytology preparation), cervical exfoliated cells collected from people of different ages and conditions, including 200 sections each with Trichomonas infection, Candida infection, herpes virus infection, and cytomegalovirus infection;

[0340] A digital scanner was used to obtain cervical cell images, and then the Unet model was used to segment the images to obtain all the cell images infected by microorganisms in the images. Data expansion was performed on the images of cells infected by various microorganisms, and finally 1,000 cells infected by each type of microorganism were obtained, and each type of microorganism infected cell image was assigned a corresponding label.

[0341] 4.2. Training microbial infected cell classifier:

[0342] The ResNet18 network was used to perform cyclic training on the microbial infected cell dataset, and the model parameters were adjusted to obtain a multi-class microbial infected cell classifier. This classifier can determine whether the input cells are infected with microorganisms and the specific type of microbial infection.

[0343] 5. Abnormal cell lesion level adjustment module, based on the patient's age, menstrual period, radiotherapy history information and microbial infection information, adjusts the weight vector ω of the relationship model between diagnostic indicators and abnormal cell lesion levels j Adjust the weights of diagnostic indicators in

[0344] The abnormal cell lesion level adjustment module includes:

[0345] The first adjustment unit determines the age of the patient. For cervical biopsy cells older than 55 years old, the nuclear-cytoplasmic ratio weight in the nuclear-cytoplasmic ratio formula is multiplied by the age influence factor λ of the nuclear-cytoplasmic ratio weight, that is, the nuclear-cytoplasmic ratio weight is adjusted;

[0346] The second adjustment unit determines the end time of the patient's menstrual period, and for cervical slice cells within 10 days after the end of the menstrual period, the mitotic weight of the nuclear-cytoplasmic ratio formula is multiplied by the menstrual influence factor η of the mitotic weight, that is, the mitotic weight is adjusted;

[0347] The third adjustment unit determines the patient's radiotherapy history information. For cervical section cells whose last radiotherapy interval is within 2 to 6 months from the cytological sampling examination, the cell degeneration change recognition model is used to determine whether the current cells have cell degeneration changes. If there are degeneration changes, the nuclear-cytoplasmic ratio weight and the vacuolar / cytoplasmic area ratio weight of the nuclear-cytoplasmic ratio formula are reset to zero.

[0348] The fourth adjustment unit makes the following adjustments according to the detection result of the cell microbial infection detection module:

[0349] If it is a Trichomonas infection, the vacuolar / cytoplasmic area ratio weight in the nuclear-cytoplasmic ratio formula is reset to zero;

[0350] If it is a Candida infection, the nuclear-to-cytoplasmic ratio weight of the nuclear-to-cytoplasmic ratio formula is reset to zero;

[0351] If it is herpes virus infection, the weights of nuclear division degree, nuclear staining uniformity, and cell crowding degree in the nuclear-cytoplasmic ratio formula are reset to zero;

[0352] If it is cytomegalovirus infection, the weights of the nuclear-cytoplasmic ratio and vacuolar / cytoplasmic area ratio in the nuclear-cytoplasmic ratio formula are reset to zero.

[0353] 6. A patient cancer level determination module determines the patient cancer level according to the lesion levels of all cells adjusted by the abnormal cell lesion level adjustment module. The specific process of determining the patient cancer level includes the following steps:

[0354] The cancer grade of patients can be divided into five categories from mild to severe, namely ASC-US, LSIL, ASC-H, HSIL, and carcinoma. Among them, carcinoma can be subdivided into SCC keratinization, SCC non-keratinization, endocervical adenocarcinoma in situ, endocervical adenocarcinoma, and endometrial adenocarcinoma.

[0355] The highest lesion level of abnormal cells in cervical cell sections is used as the patient's cancer level. If there are only atypical lesion levels, namely ASC-US and ASC-H, the number of cells of the two must reach 6 or more respectively to determine the patient's cancer level as ASC-US or ASC-H.

[0356] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0357] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. The intelligent pathological image diagnosis system based on cloud service is characterized by: The system comprises: Information acquisition module, pathological slice image preprocessing module, pathological slice image intelligent diagnosis module, preliminary diagnosis module, report generation module; The information collection module is used by information input personnel to input basic patient information and slice information in the cloud diagnosis system in sequence; The pathological slice image preprocessing module is used to acquire and process slice images to obtain slice panorama images and slice tile images; The pathological slice image intelligent diagnosis module is used to perform preliminary diagnosis and annotation on the slice tile image and output a digital pathology file; The preliminary diagnosis module is used to generate a preliminary diagnosis opinion and a preliminary analysis report based on the digital pathology file; The report generation module is used by doctors to review preliminary diagnostic opinions and preliminary analysis reports, conduct manual review and diagnosis, and generate final analysis reports; The information collection module is used by information input personnel to input basic patient information and slice information in the cloud diagnosis system in sequence; specifically: The basic information of the patient includes: the patient's age, genetic disease history, menstrual history, and previous treatment drugs; The slice information includes: slice number, slice type, staining method, and sampling time; The pathological slice image preprocessing module is used to obtain and process slice images to obtain slice panorama images and slice tile images; the implementation steps are as follows: (1) Obtaining slice scan images: Slice scan images are obtained from: 1) The microscope scanning client moves to take the slice pathology image in the current field of view, obtains the slice scan image, and transmits it to the cloud diagnosis system in real time; 2) The cloud diagnosis system supports uploading slice scans of different formats scanned by other scanning terminals on the web page; (2) stitching the slice scans into a panoramic image; The sliced ​​panorama is processed using a pyramid cutting method to obtain M×N non-overlapping small images, and the position correspondence between the non-overlapping small images and the panorama is recorded to obtain a sliced ​​tile image. The M and N represent the length and width of the non-overlapping small images respectively; The pathological slice image intelligent diagnosis module performs preliminary diagnosis and annotation on the slice tile image and outputs a digital pathology file; the specific process is as follows: (1) Determine the target detection model of the intelligent diagnosis module of the pathological slice image, use the target detection model to detect the slice tile image, and obtain abnormal cells; (2) grading the degree of cytopathic effect of abnormal cells to obtain the abnormal grade of abnormal cells; (3) Score the disease level of abnormal cells to obtain the patient's disease level; (4) Perform preliminary diagnosis and annotation based on the abnormal level of abnormal cells and the patient's condition level to obtain a digital pathology file; In (2), the abnormal cells are graded for the degree of cytopathic effect to obtain the abnormal grade of the abnormal cells; the specific process is as follows: Step 1: Obtain cell diagnostic indicators; Step 2: formulating the obtained cell diagnostic indicators; Step 3: Classify the diagnostic index levels according to the extracted diagnostic indexes; Step 4: Establish multiple sets of training data sets according to the classification levels; Step 5: Learn the numerical range of the diagnostic indicator level and determine the score of each divided level; Step 6: Fitting the relationship between each level of abnormal cells and diagnostic indicators; Step 7, calculating the probability of abnormal cells at each lesion level, and determining the lesion level of the cell; The (3) is to score the disease level of abnormal cells to obtain the disease level of the patient; the specific process is: The cell diagnostic index acquisition module acquires cell diagnostic indexes according to TBS diagnostic criteria; The cell lesion grading module is based on various diagnostic indicators of abnormal cells and uses the mixed Gaussian model M corresponding to each diagnostic indicator. i Determine the degree level of different diagnostic indicators of cells and the corresponding probability value x i , and use the relationship model between diagnostic indicators and abnormal cell lesion levels to obtain the predicted values ​​of abnormal cells at various lesion levels, and take the lesion level with the largest predicted value as the lesion level of the cell; A cell microbial infection detection module determines whether the cell image is infected with microorganisms and determines the type of microbial infection; The abnormal cell lesion level adjustment module adjusts the weight vector ω of the relationship model between the diagnostic index and the abnormal cell lesion level according to the patient's age, menstrual period, radiotherapy history information and microbial infection information. j Adjust the weights of diagnostic indicators in The patient cancer level determination module determines the patient cancer level according to the lesion level of all cells adjusted by the abnormal cell lesion level adjustment module.

2. The intelligent pathological image diagnosis system based on cloud service according to claim 1, characterized in that: The preliminary diagnosis module is used to generate a preliminary diagnosis opinion and a preliminary analysis report based on the digital pathology file; the implementation steps are as follows: Based on the digital pathology files, a preliminary diagnosis was obtained; Generate a preliminary analysis report based on the preliminary diagnosis, basic patient information, and slice information; The basic information of the patient includes: the patient's age, genetic disease history, menstrual history, and previous treatment drugs; The slice information includes: slice number, slice type, staining method, and sampling time.

3. The intelligent pathological image diagnosis system based on cloud service according to claim 2, characterized in that: The report generation module is used by doctors to review preliminary diagnosis opinions and preliminary analysis reports, conduct manual review and diagnosis, and generate final analysis reports; the implementation steps are as follows: The doctor checks the digital pathology file and preliminary analysis report, gives the final diagnosis conclusion, and generates a diagnosis report through the report generation module.

4. The intelligent pathological image diagnosis system based on cloud service according to claim 3 is characterized by: The determination process of the target detection model of the pathological slice image intelligent diagnosis module is as follows: 1) Select samples containing abnormal cells as experimental samples and enter them into the experimental sample database; 2) Label all experimental samples in the experimental sample database; specifically: A semantic segmentation model is used to segment all abnormal cells in the experimental sample database; Use a deep neural network model to classify the segmented cells; 3) The experimenter checks the annotation results in 2), corrects the wrong annotations, and completes the missing annotations to obtain the verified annotated experimental samples; 4) Input the verified labeled experimental samples into different target detection models for training to obtain different trained target detection models; Calculate the accuracy, precision, recall, F1-Measure, and loss value of inputting the verified labeled experimental samples into different trained target detection models; The target detection model with the highest accuracy is selected as the target detection model of the intelligent diagnosis module of pathological slice images.