A quality control method and system for pathological sections based on target detection

By constructing a multi-scale and multi-receptive field adaptive rotation object detection algorithm, the problem of low detection accuracy and efficiency in the field of pathological slice quality control is solved, and the automated quality control score of pathological slices is realized, which improves detection accuracy and efficiency .

CN114494108BActive Publication Date: 2024-11-26北京知见生命科技有限公司 +2
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202111347784.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-15
Publication Date
2024-11-26
Estimated Expiration
2041-11-15

AI Technical Summary

Technical Problem

The existing technology is difficult to apply the object detection algorithm to the field of quality control of pathological sections. It is mainly due to the problems of knife marks, holes, wrinkles, contamination and micro-calcification of the pathological sections, which leads to low detection accuracy and efficiency, and takes up a lot of time for the doctor. Repeat work.

Method used

A quality control model including convolutional neural network, feature pyramid network, multi-scale receptive field network and four branch networks was constructed. The target categories and central points of pathological slices were extracted through the adaptive rotation object detection algorithm of multi-scale and multi-receptive field. The coordinates, length and width and rotation angle are constructed to carry out iterative training to achieve automatic quality control scores for pathological slices.

Benefits of technology

It realizes automated quality control of pathological sections, improves detection accuracy and efficiency, reduces doctors' time occupancy in repetitive work, and can accurately evaluate the quality of sections, and has broad application prospects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114494108B_ABST
    Figure CN114494108B_ABST
Patent Text Reader

Abstract

The present invention proposes a quality control method and system for pathological sections based on target detection. The present invention uses a target detection algorithm based on key points, so there is no need to set a candidate frame, and the position of the target detection frame is directly output. At the same time, the present invention also adds a detection frame rotation network, which can better fit the position of knife marks or holes, so as to accurately obtain the size of holes, knife marks and other categories on the slice, so as to accurately score and control the slice.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image classification in artificial intelligence, and in particular to a quality control method and system for pathological sections based on target detection. Background Art

[0002] Pathological sections are sections of pathological tissue of a certain size made using pathological histological methods. The quality of pathological sections plays an important role in pathological diagnosis, so it is particularly important to strive to improve the quality of pathological sections. However, the current quality control of pathological sections still remains at the stage where doctors conduct random inspections and scoring of sections to find out the problems with the sections, which takes up a lot of doctors' time for repetitive work.

[0003] With the rapid development of convolutional neural networks in the field of target detection, the accuracy and speed of target detection algorithms have been qualitatively improved. At present, target detection algorithms based on deep learning are mainly divided into two categories: candidate box-based and key point-based target detection algorithms. Among the candidate box-based algorithms, the more classic algorithms include R-CNN, SSD, etc. The R-CNN algorithm first applied convolutional neural networks to target detection, and obtained the final target position by screening a large number of candidate regions. The SSD target detection algorithm introduces multi-scale feature maps, selects different numbers of candidate box regions on each feature map, and finally outputs the detection results through the network regression algorithm. The target detection algorithms based on key points mainly include FCOS, CornerNet, etc. FCOS obtains the target position area by predicting the center point of the target area and the length of the target detection box in the four directions of top, bottom, left, and right. CornerNet obtains the coordinate position of the target by predicting the coordinate positions of the upper left and upper right corners of the target area.

[0004] Although there are many object detection algorithms based on convolutional neural networks, it is difficult to apply object detection algorithms to the field of pathological slice quality control. The main reasons include the difficulty in collecting problematic pathological slices. The main problems with current pathological slices are: knife marks, holes, wrinkles, contamination, and tissue microcalcification. The exact location of these problematic slices is detected by the object detection algorithm, and the quality of the slice is judged by the size of the detection frame. For example, if a knife mark is detected, the severity of the knife mark is judged based on the length and width of the detection frame (knife marks that are too long or too wide will affect the quality of the slice). If a hole is detected, the size of the hole is judged by the detection frame, so as to perform quality control scoring on the pathological slice. Summary of the invention

[0005] Based on the above problems, the present invention proposes a quality control method for pathological sections based on target detection, comprising:

[0006] Step 1: construct a quality control model including a convolutional neural network, a feature pyramid network, a multi-scale receptive field network and four branch networks to obtain multiple original pathological slice images with labeled target categories, target center point coordinates, target length and width, and target rotation angle;

[0007] Step 2: extracting a basic feature map of the original pathological slice image through the convolutional neural network, inputting the basic feature map into the feature pyramid network to obtain pyramid feature maps of multiple scales, and inputting the pyramid feature map into the multi-scale receptive field network to obtain a multi-scale receptive field feature map;

[0008] Step 3: Input the multi-receptive field feature map into four branch networks, and output the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image respectively;

[0009] Step 4: Based on the target category, target center point coordinates, target length and width, and target rotation angle annotated on the original pathological slice image, and the target category, target center point coordinates, target length and width, and target rotation angle detected on the original pathological slice image, a loss function is constructed to iteratively train the quality control model until the loss function converges or reaches a preset number of iterations, and the current quality control model is saved as the final recognition model;

[0010] Step 5: Extract the target category, target center point coordinates, target length and width, and target rotation angle of the pathological slice image to be quality controlled through the final recognition model.

[0011] The method for quality control of pathological sections based on target detection includes:

[0012] Step 6: According to the target length and width, target center point coordinates and target rotation angle of the pathological slice image to be quality controlled, the target area corresponding to each target category to which the pathological slice image to be quality controlled belongs is obtained to score the pathological slice image to be quality controlled. If the score is lower than the threshold, the pathological slice to be quality controlled is judged to be unqualified.

[0013] In the target detection-based quality control method for pathological sections, the target categories include: knife marks and / or cavities and / or wrinkles and / or contamination and / or tissue microcalcification.

[0014] The quality control method of pathological sections based on target detection, wherein the convolutional neural network adopts the ResNet-50 network structure; the feature pyramid network adopts a multi-scale long-link feature pyramid network structure; the four branch networks are respectively a category branch, a target center point branch, a target length and width branch, and a target rotation angle branch network.

[0015] The present invention also proposes a quality control system for pathological sections based on target detection, which includes:

[0016] A model building module is used to build a quality control model including a convolutional neural network, a feature pyramid network, a multi-scale receptive field network, and four branch networks to obtain multiple original pathological slice images with labeled target categories, target center point coordinates, target length and width, and target rotation angle;

[0017] A feature extraction module is used to enable the convolutional neural network to extract a basic feature map of the original pathological slice image, input the basic feature map into the feature pyramid network to obtain pyramid feature maps of multiple scales, and input the pyramid feature map into the multi-scale receptive field network to obtain a multi-scale receptive field feature map;

[0018] A detection module is used to input the multi-receptive field feature map into four branch networks, and respectively output the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image;

[0019] A training module, for constructing a loss function based on the target category, target center point coordinates, target length and width, and target rotation angle annotated by the original pathological slice image, and the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image, so as to iteratively train the quality control model until the loss function converges or reaches a preset number of iterations, and save the current quality control model as the final recognition model;

[0020] The recognition module is used to extract the target category, target center point coordinates, target length and width, and target rotation angle of the pathological slice image to be quality controlled through the final recognition model.

[0021] The quality control system of pathological sections based on target detection includes:

[0022] The quality evaluation module is used to obtain the target area corresponding to each target category of the pathological slice image to be quality controlled according to the target length and width, target center point coordinates and target rotation angle of the pathological slice image to be quality controlled, so as to score the pathological slice image to be quality controlled. If the score is lower than the threshold, the pathological slice to be quality controlled is judged to be unqualified.

[0023] The quality control system of pathological sections based on target detection, wherein the target categories include: knife marks and / or cavities and / or wrinkles and / or contamination and / or tissue microcalcification.

[0024] The quality control system of pathological sections based on target detection, wherein the convolutional neural network adopts the ResNet-50 network structure; the feature pyramid network adopts a multi-scale long-link feature pyramid network structure; the four branch networks are respectively a category branch, a target center point branch, a target length and width branch, and a target rotation angle branch network.

[0025] The present invention also proposes a storage medium for storing a program for executing any one of the pathological section quality control methods based on target detection.

[0026] The present invention also proposes a client, which is used for any of the quality control systems of pathological sections based on target detection.

[0027] It can be seen from the above scheme that the advantages of the present invention are:

[0028] The present invention proposes a multi-scale, multi-receptive field-based target detection algorithm to perform quality control scoring on pathological sections. The position of the adaptively rotated detection frame of the target is output through the constructed target detection network. Therefore, target detection and pathological section quality control are combined, which has a wide range of application prospects in the field of pathological section quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the method of the present invention;

[0030] Figure 2 This is a network structure diagram of the present invention. DETAILED DESCRIPTION

[0031] The present invention proposes a pathological section quality control algorithm based on target detection. Because the sizes of problems such as knife marks and holes vary greatly, if an algorithm based on a detection frame is used, it is necessary to pre-set the running speed of many candidate frames of different scales. The present invention uses a target detection algorithm based on key points, so there is no need to set a candidate frame, and the position of the target detection frame is directly output. At the same time, the present invention also adds a detection frame rotation network, which can better fit the position of knife marks or holes, so as to accurately obtain the size of holes, knife marks and other categories on the slice, so as to accurately score and quality control the slice.

[0032] The present invention aims to solve the above problems of the prior art and proposes an adaptive rotation target detection algorithm based on multi-scale and multi-receptive fields. The technical solution of the present invention is as follows:

[0033] A pathological section quality control algorithm based on target detection includes the following steps:

[0034] S1. First, obtain the pathological slices, preprocess the pathological slices, and then input them into the basic network (convolutional network) structure to extract the basic general feature map.

[0035] S2. Secondly, the basic universal feature map obtained in S1 is passed to the feature pyramid network structure to obtain pyramid feature maps of five different scales. The function of the feature pyramid is to introduce multi-scale information. Because features of different scales correspond to different target sizes. Generally, large feature maps correspond to small targets, and small feature maps correspond to large targets. By adding feature maps of different scales through the feature pyramid, large and small targets can be better detected.

[0036] S3. Again, the pyramid feature maps of each scale are transmitted to the multi-scale receptive field network structure to obtain feature maps of receptive fields of different scales.

[0037] S4. Finally, the multi-receptive field feature map obtained in S3 is input into four branch networks to output the category of the detected target, the coordinates of the center point of the target, the length and width of the target, and the rotation angle of the target. The categories of the target may include knife marks, cavities, wrinkles, contamination, and tissue microcalcification.

[0038] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0039] Step S1: First obtain the pathological slice, scale the original pathological slice to a size of 512×512, and input the scaled image into the ResNet-50 network structure to extract the basic general feature map.

[0040] Y=F(x) (1)

[0041] In formula (1), x represents the scaled medical image, F represents the ResNet-50 network structure, and Y represents the extracted basic general feature map.

[0042] Step S2: Input the general feature map obtained in step S1 into the multi-scale long link feature pyramid to obtain multi-scale feature information.

[0043]

[0044] In formula (2), S represents the multi-scale feature information, and Z represents the output result of each layer of the feature pyramid.

[0045] Step S3: Input the multi-scale feature map into the multi-receptive field network structure, and extract multi-receptive field feature information from the feature pyramid feature maps of five different scales.

[0046]

[0047] In formula (3), T represents the multi-receptive field network structure. i represents the number of layers corresponding to the multi-scale receptive field network structure, and j represents the number of layers corresponding to the feature pyramid.

[0048] Step S4: Finally, the multi-scale, multi-receptive field feature map obtained in S3 is input into the prediction network structure to output the location and category of the target. The prediction network structure consists of four branches: category branch, target center point branch, target length and width branch, and target rotation angle branch.

[0049] After the detection frame is obtained through step S4, it can be judged based on the size of the detection frame, for example: the number of knife marks in the slice and the size of each knife mark. The current pathological slice is scored based on the number and size of the detected knife marks. If the score is lower than a certain score, it is considered to be an unqualified pathological slice, which will remind the doctor who makes the slice of the disease, thereby ensuring the quality assurance of subsequent slice making.

[0050] For example, the standard is that if the knife mark is within 1mm, no points will be deducted, if it is within 1-5mm, 1 point will be deducted, if it is 5-10mm, 2 points will be deducted, and if it exceeds 10mm, it will be directly unqualified. Two knife marks were detected in a slice, the lengths were 2mm and 6mm respectively, and the full score was 10 points. He was deducted 3 points, and the slice scored 7 points, which was barely qualified.

[0051] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0052] The present invention also proposes a quality control system for pathological sections based on target detection, which includes:

[0053] A model building module is used to build a quality control model including a convolutional neural network, a feature pyramid network, a multi-scale receptive field network, and four branch networks to obtain multiple original pathological slice images with labeled target categories, target center point coordinates, target length and width, and target rotation angle;

[0054] A feature extraction module is used to enable the convolutional neural network to extract a basic feature map of the original pathological slice image, input the basic feature map into the feature pyramid network to obtain pyramid feature maps of multiple scales, and input the pyramid feature map into the multi-scale receptive field network to obtain a multi-scale receptive field feature map;

[0055] A detection module is used to input the multi-receptive field feature map into four branch networks, and respectively output the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image;

[0056] A training module, for constructing a loss function based on the target category, target center point coordinates, target length and width, and target rotation angle annotated by the original pathological slice image, and the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image, so as to iteratively train the quality control model until the loss function converges or reaches a preset number of iterations, and save the current quality control model as the final recognition model;

[0057] The recognition module is used to extract the target category, target center point coordinates, target length and width, and target rotation angle of the pathological slice image to be quality controlled through the final recognition model.

[0058] The quality control system of pathological sections based on target detection includes:

[0059] The quality evaluation module is used to obtain the target area corresponding to each target category of the pathological slice image to be quality controlled according to the target length and width, target center point coordinates and target rotation angle of the pathological slice image to be quality controlled, so as to score the pathological slice image to be quality controlled. If the score is lower than the threshold, the pathological slice to be quality controlled is judged to be unqualified.

[0060] The quality control system of pathological sections based on target detection, wherein the target categories include: knife marks and / or cavities and / or wrinkles and / or contamination and / or tissue microcalcification.

[0061] The quality control system of pathological sections based on target detection, wherein the convolutional neural network adopts the ResNet-50 network structure; the feature pyramid network adopts a multi-scale long-link feature pyramid network structure; the four branch networks are respectively a category branch, a target center point branch, a target length and width branch, and a target rotation angle branch network.

[0062] The present invention also proposes a storage medium for storing a program for executing any one of the pathological section quality control methods based on target detection.

[0063] The present invention also proposes a client, which is used for any of the quality control systems of pathological sections based on target detection.

Claims

1. A quality control method for pathological sections based on target detection, It is characterized in that include: Step 1: construct a quality control model including a convolutional neural network, a feature pyramid network, a multi-scale receptive field network and four branch networks to obtain multiple original pathological slice images with labeled target categories, target center point coordinates, target length and width, and target rotation angle; Step 2: extracting a basic feature map of the original pathological slice image through the convolutional neural network, inputting the basic feature map into the feature pyramid network to obtain pyramid feature maps of multiple scales, and inputting the pyramid feature map into the multi-scale receptive field network to obtain a multi-scale receptive field feature map; Step 3: Input the multi-scale receptive field feature map into four branch networks, and output the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image respectively; Step 4: Based on the target category, target center point coordinates, target length and width, and target rotation angle annotated on the original pathological slice image, and the target category, target center point coordinates, target length and width, and target rotation angle detected on the original pathological slice image, a loss function is constructed to iteratively train the quality control model until the loss function converges or reaches a preset number of iterations, and the current quality control model is saved as the final recognition model; Step 5: extracting the target category, target center point coordinates, target length and width, and target rotation angle of the pathological slice image to be quality controlled through the final recognition model; Step 6: According to the target length and width, target center point coordinates and target rotation angle of the pathological slice image to be quality controlled, the target area corresponding to each target category to which the pathological slice image to be quality controlled belongs is obtained to score the pathological slice image to be quality controlled. If the score is lower than the threshold, the pathological slice to be quality controlled is judged to be unqualified.

2. The quality control method of pathological sections based on target detection as claimed in claim 1, It is characterized in that The target categories include: cuts and / or cavities and / or wrinkles and / or contamination and / or tissue microcalcifications.

3. The quality control method of pathological sections based on target detection as claimed in claim 1, It is characterized in that The convolutional neural network adopts the ResNet-50 network structure; The feature pyramid network adopts a multi-scale long-link feature pyramid network structure; the four branch networks are respectively a category branch, a target center point branch, a target length and width branch, and a target rotation angle branch network.

4. A quality control system for pathological sections based on target detection, It is characterized in that include: A model building module is used to build a quality control model including a convolutional neural network, a feature pyramid network, a multi-scale receptive field network, and four branch networks to obtain multiple original pathological slice images with labeled target categories, target center point coordinates, target length and width, and target rotation angle; A feature extraction module is used to enable the convolutional neural network to extract a basic feature map of the original pathological slice image, input the basic feature map into the feature pyramid network to obtain pyramid feature maps of multiple scales, and input the pyramid feature map into the multi-scale receptive field network to obtain a multi-scale receptive field feature map; A detection module is used to input the multi-scale receptive field feature map into four branch networks, and respectively output the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image; A training module, for constructing a loss function based on the target category, target center point coordinates, target length and width, and target rotation angle annotated by the original pathological slice image, and the target category, target center point coordinates, target length and width, and target rotation angle detected by the original pathological slice image, so as to iteratively train the quality control model until the loss function converges or reaches a preset number of iterations, and save the current quality control model as the final recognition model; A recognition module, used for extracting the target category, target center point coordinates, target length and width, and target rotation angle of the pathological slice image to be quality controlled through the final recognition model; The quality evaluation module is used to obtain the target area corresponding to each target category of the pathological slice image to be quality controlled according to the target length and width, target center point coordinates and target rotation angle of the pathological slice image to be quality controlled, so as to score the pathological slice image to be quality controlled. If the score is lower than the threshold, the pathological slice to be quality controlled is judged to be unqualified.

5. The quality control system for pathological sections based on target detection as claimed in claim 4, It is characterized in that The target categories include: cuts and / or cavities and / or wrinkles and / or contamination and / or tissue microcalcifications.

6. The quality control system for pathological sections based on target detection as claimed in claim 5, It is characterized in that The convolutional neural network adopts the ResNet-50 network structure; The feature pyramid network adopts a multi-scale long-link feature pyramid network structure; the four branch networks are respectively a category branch, a target center point branch, a target length and width branch, and a target rotation angle branch network.

7. A storage medium for storing a program for executing the quality control method of pathological sections based on target detection as claimed in any one of claims 1 to 3.

8. A client, used for any one of the pathological section quality control systems based on target detection in claims 4 to 6.

Citation Information

Cited By

  • Rapid focusing method for scanning pathological image based on lightweight network

    CN117455791A