Pathological image quality control method and device based on deep learning, equipment and medium

Abstract

The embodiment of the specification discloses a pathological image quality control method and device based on deep learning, equipment and medium. The method comprises the following steps: determining a plurality of classification layers of a preset deep learning model according to a preset quality control condition of a pathological image and a business work corresponding to the pathological image; constructing a data set according to a first pathological image without disease and a second pathological image with disease; the data set comprises a training set, a verification set and a test set; inputting the first pathological image and the second pathological image contained in the training set as input images into a preset deep learning model, iteratively training the preset deep learning model, verifying the preset deep learning model based on the verification set, and obtaining a quality control model to be tested; testing the quality control model to be tested based on the test set to obtain a quality control model meeting the requirements; and inputting a to-be-detected pathological image into the quality control model meeting the requirements to obtain a quality control result of the to-be-detected pathological image.

Description

TECHNICAL FIELD

[0001] The present specification relates to the field of pathological image processing, and in particular to a pathological image quality control method and device based on deep learning, and a medium. BACKGROUND

[0002] Pathological sections can be scanned into digital pathological images by a digital scanner and stored in a computer, and can be read and analyzed by a computer program. Digital pathology optimizes the pathological workflow and provides a basis for computer-aided diagnosis. However, due to some unpredictable non-human adverse effects in the process of making pathological sections, such as artifacts, bubbles, dust, broken slides, and tissue overlap, or due to different factors such as section processing environment, reagent dosage, and operator, the results of the sections are not consistent, so pathological quality control of pathological images has become an important part of the medical field.

[0003] The existing method of manual discrimination for identifying and classifying pathological images and the labeling process is complex. In order to solve the problems of manual discrimination, the existing related technology can train a model for labeling and classification based on a certain amount of labeling results, and then use the trained model to automatically perform image quality control and image classification on new input images. However, due to the influencing factors in the section processing process, it is easy to cause inconsistent results of the sections, making it impossible to guarantee the stability and robustness of the computer-aided diagnosis software, and due to the diversity of pathological images, the quality control factors of each pathological image are different, making it impossible to effectively perform efficient image quality control on each pathological image. SUMMARY

[0004] One or more embodiments of the present specification provide a pathological image quality control method, device, equipment and medium based on deep learning, to solve the technical problem of how to provide an efficient image quality control method for pathological images.

[0005] One or more embodiments of the present specification adopt the following technical solutions:

[0006] One or more embodiments of the present specification provide a pathological image quality control method based on deep learning, the method comprising:

[0007] Based on the preset quality control conditions of the pathological image and the business work corresponding to the pathological image, a plurality of classification layers of a preset deep learning model are determined;

[0008] According to the collected first pathological image without disease and the second pathological image with disease, a data set corresponding to the preset deep learning model is constructed; wherein the data set includes: a training set, a validation set, and a test set;

[0009] inputting the first pathological image and the second pathological image contained in the training set as input images, inputting the preset deep learning model, iteratively training the preset deep learning model, verifying the preset deep learning model of each iteration based on the verification set, to obtain a quality control model to be tested;

[0010] testing the quality control model to be tested based on the test set, to obtain a quality control model meeting the requirements;

[0011] inputting the pathological image to be detected into the quality control model meeting the requirements, to obtain a quality control result of the pathological image to be detected.

[0012] Further, in one or more embodiments of the present specification, before the preset quality control condition based on the pathological image and the business work corresponding to the pathological image determine the multiple classification layers of the preset deep learning model, the method further comprises:

[0013] inputting the network with the corresponding pre-training network on the data set as the backbone network of the preset deep learning model;

[0014] modifying the parameters of the classification layer of the backbone network according to the preset quality control condition corresponding to the preset deep learning model, to obtain a modified classification layer, and adding the classification layer of the backbone network to obtain a new classification layer; wherein the parameter is the number of classification categories;

[0015] transferring the features of the feature layer located before the classification layer to the modified classification layer and the new classification layer, to construct the preset deep learning model based on the backbone network containing the modified classification layer and the new classification layer.

[0016] Further, in one or more embodiments of the present specification, the preset quality control condition based on the pathological image and the business work corresponding to the pathological image determine the multiple classification layers of the preset deep learning model, specifically comprising:

[0017] dividing the preset deep learning model into main category classification layers according to the business work corresponding to the pathological image;

[0018] determining multiple fine classification layers of the preset deep learning model according to the disease types and attributes of the pathological images of each main classification layer, and adding the preset quality control condition corresponding to each fine classification layer to the fine classification layer;

[0019] generating multiple classification layers of the preset classification model according to the main category classification layer and the fine classification layer.

[0020] Further, the data set corresponding to the preset deep learning model is constructed according to the collected first pathological image without disease and the second pathological image with disease, and specifically includes:

[0021] The first pathological image without disease is collected as a natural image subset, and the first pathological image with disease is collected as a pathological image subset;

[0022] The data set of the preset deep learning model is constituted based on the natural image subset and the pathological image subset, and the data set is proportionally divided into a training set, a tuning set and a test set.

[0023] Further, the first pathological image and the second pathological image contained in the training set are taken as input images, the preset deep learning model is input, the preset deep learning model is iteratively trained, and the preset deep learning model of each iteration is verified based on the verification set to obtain a quality control model to be tested, and specifically includes:

[0024] The first pathological image and the second pathological image contained in the training set are taken as input images;

[0025] The input images are preprocessed based on a preset mode to obtain processed input images; wherein the preset mode includes upsampling, random rotation, color jittering, dye enhancement, random horizontal, vertical flipping;

[0026] The processed input images are input into the preset deep learning model to obtain a first prediction result of the preset deep learning model, and the first prediction result is compared with a label value of the input image to obtain a first comparison result;

[0027] According to the first comparison result and a preset loss function of the preset deep learning model, a first loss function value of each classification layer of the preset deep learning model is determined, and a second loss function value of the preset deep learning model is determined based on the first loss function value;

[0028] According to the second loss function value and a preset hyperparameter of the preset deep learning model, gradient descent and parameter updating are performed on the preset deep learning model to obtain an updated preset deep learning model;

[0029] The updated preset deep learning model is verified based on the verification set, and a third loss function value of the updated preset deep learning model obtained according to the verification set is obtained, and the iteration update of the preset deep learning model is ended to obtain a quality control model to be tested.

[0030] Further, the updated preset deep learning model is verified based on the verification set, and a third loss function value of the updated preset deep learning model obtained based on the verification set is used to end the iterative updating of the preset deep learning model to obtain a quality control model to be tested, specifically including:

[0031] The first pathological image and the second pathological image in the verification set are input into the updated preset deep learning model to obtain a second prediction result of the updated preset deep learning model.

[0032] A second comparison result is obtained by comparing the second prediction result with label values of the first pathological image and the second pathological image.

[0033] A third loss function value of each classification layer of the updated preset deep learning model is determined based on the first comparison result and a preset loss function of the updated preset deep learning model, and a fourth loss function value of the updated preset deep learning model is determined based on the third loss function value.

[0034] If it is determined that the fourth loss function value increases or remains unchanged, the iterative updating of the preset deep learning model is ended, and the updated preset deep learning model is used as a quality control model to be tested.

[0035] Further, the preset hyperparameters of the preset deep learning model include a learning rate and an optimization algorithm.

[0036] One or more embodiments of the present specification provide a pathological image quality control device based on deep learning, and the device includes:

[0037] A determination module is configured to determine a plurality of classification layers of a preset deep learning model based on preset quality control conditions of a pathological image and business work corresponding to the pathological image.

[0038] A construction module is configured to construct a data set corresponding to the preset deep learning model based on collected first pathological images without diseases and second pathological images with diseases, wherein the data set includes a training set, a verification set, and a test set.

[0039] A training and verification module is configured to input first pathological images and second pathological images included in the training set as input images into the preset deep learning model, iteratively train the preset deep learning model, and verify the preset deep learning model at each iteration based on the verification set to obtain a quality control model to be tested.

[0040] A test module is configured to test the quality control model to be tested based on the test set to obtain a quality control model meeting requirements.

[0041] The acquisition module is configured to input a to-be-detected pathological image into the qualified quality control model to obtain a quality control result of the to-be-detected pathological image.

[0042] One or more embodiments of the present specification provide a pathological image quality control device based on deep learning, the device comprising:

[0043] at least one processor; and

[0044] a memory in communication connection with the at least one processor; wherein

[0045] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0046] determine a plurality of classification layers of a preset deep learning model based on a preset quality control condition of a pathological image and a business work corresponding to the pathological image;

[0047] construct a data set corresponding to the preset deep learning model according to a first pathological image without disease and a second pathological image with disease collected; wherein the data set comprises a training set, a validation set, and a test set;

[0048] input the first pathological image and the second pathological image contained in the training set as input images into the preset deep learning model, iteratively train the preset deep learning model, and verify the preset deep learning model at each iteration based on the validation set to obtain a to-be-tested quality control model;

[0049] test the to-be-tested quality control model based on the test set to obtain a qualified quality control model;

[0050] input a to-be-detected pathological image into the qualified quality control model to obtain a quality control result of the to-be-detected pathological image.

[0051] One or more embodiments of the present specification provide a non-volatile computer storage medium storing computer executable instructions, the computer executable instructions being configured to:

[0052] determine a plurality of classification layers of a preset deep learning model based on a preset quality control condition of a pathological image and a business work corresponding to the pathological image;

[0053] construct a data set corresponding to the preset deep learning model according to a first pathological image without disease and a second pathological image with disease collected; wherein the data set comprises a training set, a validation set, and a test set;

[0054] input the first pathological image and the second pathological image contained in the training set as input images, input the preset deep learning model, iteratively train the preset deep learning model, verify the preset deep learning model of each iteration based on the verification set, and obtain a quality control model to be tested;

[0055] test the quality control model to be tested based on the test set, and obtain a quality control model meeting the requirements;

[0056] input the pathological image to be detected into the quality control model meeting the requirements, and obtain the quality control result of the pathological image to be detected.

[0057] The above at least one technical solution adopted by the embodiments of the present specification can achieve the following beneficial effects: by dividing and adding multiple classification layers, the attributes of the image are constrained, which can distinguish the categories of pathological images and distinguish pathological images from normal images, and further according to the quality control conditions, the quality control discrimination of pathological images is realized, solving the problems of low recognition accuracy caused by different sheet effects due to unpredictable non-human adverse effects in existing pathological images and the problem of not fine classification caused by different attributes of pathological images. By gradient descent and parameter update after each training of the preset deep learning model, the model is iteratively optimized, and the updated preset deep learning model is verified based on the verification set. Based on the end update strategy, the quality control model to be tested is locked, so that the quality control model to be tested is tested based on the test set, and a quality control model meeting the requirements is obtained. The recognition accuracy of the quality control model is improved. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0059] Figure 1 A method flowchart of a pathological image quality control method based on deep learning provided by an embodiment of the present specification;

[0060] Figure 2 A network structure diagram under an application scenario based on multiple classification layers provided by an embodiment of the present specification;

[0061] Figure 3 An internal structure diagram of a pathological image quality control device based on deep learning provided by an embodiment of the present specification;

[0062] Figure 4 FIG. 1 is a schematic diagram of an internal structure of a deep learning-based pathological image quality control device according to an embodiment of the present specification;

[0063] Figure 5 FIG. 2 is a schematic diagram of an internal structure of a non-volatile storage medium according to an embodiment of the present specification. DETAILED DESCRIPTION

[0064] The embodiments of the present specification provide a deep learning-based pathological image quality control method, device, equipment and medium.

[0065] Pathological images contain rich phenotypic information, which can be used for diagnosis and monitoring of cancer patients, predicting survival rate, and providing reference for personalized cancer treatment. Traditional pathological diagnosis is completed by evaluating the living tissue sections on the glass slides through a microscope. However, this process is often tedious and time-consuming, and most areas are facing a serious shortage of pathologists. At the same time, affected by the experience level and subjective factors of pathologists, the analysis results are prone to have large differences, and even cause false negative diagnosis results, thereby affecting the treatment plan of patients. In addition, artificial visual evaluation also hinders the reproducibility of the analysis results, especially not conducive to the judgment of indicators that rely heavily on quantitative histopathology.

[0066] The rapid development of digital microscopes brings opportunities for digital pathology image analysis, realizes the digital management of patient pathological sections, and can reduce the identification errors of sections, and reduce the risk of damage and loss. However, due to some unpredictable non-human adverse effects in the process of making pathological sections, such as artifacts, bubbles, dust, broken slides and tissue overlap, etc. In addition, due to different factors such as section processing environment, reagent dosage and operating personnel, the effects of the sections are inconsistent, thereby causing large errors in the identification results. And the types and attributes of pathological images are more, the existing classification method cannot realize the image quality control of pathological images of multiple types and multiple attributes under different quality control conditions respectively. In order to solve the above technical problems, the embodiments of the present specification provide a deep learning-based pathological image quality control method, device, equipment and medium.

[0067] In order for those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be described clearly and completely in conjunction with the drawings in the embodiments of the present specification. Obviously, the described embodiments are only part of the embodiments of the present specification, not all embodiments. Based on the embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present specification.

[0068] As Figure 1The one or more embodiments of the present specification provide a flowchart of a deep learning-based pathological image quality control method. The method comprises the following steps: Figure 1 It can be seen that the method comprises the following steps:

[0069] S101: determining a plurality of classification layers of a preset deep learning model based on a preset quality control condition of a pathological image and a business work corresponding to the pathological image.

[0070] In order to achieve fine division of the pathological image, according to the preset quality control condition of the pathological image and the business work corresponding to the pathological image, the plurality of classification layers of the preset deep learning model are determined in the embodiments of the present specification, so as to achieve fine identification of the pathological image based on the plurality of classification layers.

[0071] In one or more embodiments of the present specification, in order to refine the identification attribute of the pathological image corresponding to the classification layer, before determining the plurality of classification layers of the preset deep learning model based on the preset quality control condition of the pathological image and the business work corresponding to the pathological image, the method further comprises the following steps:

[0072] The network with the corresponding pre-training network on the data set is taken as the backbone network of the preset deep learning model. According to the preset quality control condition corresponding to the preset deep learning model, the parameters of the classification layer of the backbone network are modified to obtain the modified classification layer, and the classification layer of the backbone network is added to obtain the new classification layer; it should be noted that the parameter is the number of classification categories. The features of the feature layer located before the classification layer are transmitted to the modified classification layer and the new classification layer, so as to construct the preset deep learning model based on the backbone network containing the modified classification layer and the new classification layer.

[0073] Specifically, an additional class is added in the original classification layer in the preset deep learning model, which is the pathological image class. An additional classification layer is added, which is used for the binary classification of natural images and pathological images. Then, according to the quality control requirements, additional specific classification layers are added, such as tumor site classification layer, magnification layer, whether it meets the quality control layer, and the specific reason for not meeting the quality control layer. In some specific layers, the "not applicable" class needs to be added, which is a classification category for non-pathological images, to realize the construction of multiple classification layers. It should be noted that the selection of the backbone network needs to meet the corresponding pre-training network on the selected data set, for example: AlexNet, VGG series, ResNet series, SqueezeNet, DenseNet series, Inception series, GoogLeNet, ShuffleNet series, MobileNet series, ResNeXt series, MNASNet, EffcientNet, RegNet, VisionTransformer series, ConvNeXt series and SwinTransformer series, etc. and their variants. On the basis of the selected backbone network, we add or modify the quality control classification layer or regression layer defined by ourselves. In one application scenario, we can choose to use the swin_base_patch4_window12_384_in22k network structure. According to the requirements of the quality control layer defined in this application scenario, first change the out_features parameter of the Linear in the head classification layer of this network structure from 21841 to 21842, and then add other head layers (classification layers), the specific out_features parameter of the Linear in the head layer (classification layer) is the number of classification categories defined above, which is 2, 9, 8, 3 and 6. The features of the feature layer before the original head layer (classification layer) will be changed to be transmitted to these modified and newly added quality control layers at the same time.

[0074] In one or more embodiments of the specification, based on the preset quality control conditions of the pathological image and the business work corresponding to the pathological image, a plurality of classification layers of the preset deep learning model are determined, specifically including:

[0075] According to the business work corresponding to the pathological image, the preset deep learning model is divided into a main category classification layer. For example, if the business work is to identify whether the pathological image has a disease, the category classification layer can be divided into "pathological image" and "natural image". Then, according to the disease type and attribute of the pathological image corresponding to each main classification layer, a plurality of fine classification layers of the preset deep learning model are determined. For example, according to the type, the "pathological image" is refined into fine classification layers such as breast cancer, gastric cancer, lung cancer, prostate cancer, endometrial cancer, cervical cancer, and thyroid cancer. The preset quality control conditions corresponding to each fine classification layer are added to the respective fine classification layer, and according to the main category classification layer and the fine classification layer, a plurality of classification layers of the preset classification model are generated.

[0076] Specifically, the classification layer is mainly divided into a main category classification layer and a fine classification layer. When the first pathological image or the second pathological image is not applicable in a specific layer, a specific category "not applicable" needs to be added to ensure the convergence of the model. The main category classification layer mainly distinguishes between large categories, such as Figure 2 as shown, for distinguishing between the categories "pathological image" and "natural image"; the fine classification layer is for distinguishing between specific image categories or attributes, such as distinguishing between which type of natural image and which part the pathological image comes from. As shown in Figure 2 as shown, in one application scenario of the embodiments of the present specification, according to the specific quality control requirements and business needs, the classification layer is defined as follows:

[0077] One classification layer is used to distinguish between specific natural images and pathological images, and the number of classification categories is 21841+1 (according to the selected natural image dataset, in this example, ImageNet 22k); one classification layer is used to distinguish between pathological images and natural images, and the number of classification categories is 2. One classification layer is used to distinguish between specific cancer types, such as breast cancer, gastric cancer, lung cancer, prostate cancer, endometrial cancer, cervical cancer, thyroid cancer, other categories, and not applicable, and the number of classification categories is 9. One classification layer is used to distinguish between image magnifications, such as 1x, 2x, 4x, 10x, 20x, 40x magnification, and other magnifications and not applicable, and the number of classification categories is 8. This layer can also be used as a regression layer, and the prediction range is 0-40. One classification layer is used to distinguish whether it meets the quality control, and the number of classification categories is 3, which distinguishes between meeting the quality control, not meeting the quality control, and not applicable. One classification layer is used to distinguish the specific reasons for not meeting the quality control, and the number of classification categories is 6, which distinguishes between over-staining, tissue overlap, cracks, scratches, and other reasons and not applicable.

[0078] S102: According to the collected first pathological image without disease and second pathological image with disease, a data set corresponding to the preset deep learning model is constructed; wherein the data set includes: a training set, a validation set, and a test set.

[0079] In order to improve the recognition accuracy of the model, the preset deep learning model needs to be trained based on the data set. In one or more embodiments of the present specification, according to the collected first pathological image without disease and the second pathological image with disease, a data set corresponding to the preset deep learning model is constructed, which specifically includes the following steps: first, the first pathological image without disease is collected as a natural image subset, and the first pathological image with disease is collected as a pathological image subset. Then, the data set of the preset deep learning model is constructed based on the natural image subset and the pathological image subset, and the data set is divided into a training set, a tuning set and a test set in proportion. It should be noted that in terms of data set division, the data set needs to be divided into a training set, a validation set and a test set in proportion, and the sample distribution needs to be as balanced as possible and without intersection between each other, that is, the images from the same patient or from the same slice only appear in a certain set and do not appear in other sets at the same time. In the application scenario of the first embodiment of the present specification, the training set, the validation set and the test set can be divided in the proportion of 2:1:2 according to the tasks of each classification layer and each subset.

[0080] S103: input the first pathological image and the second pathological image contained in the training set as input images into the preset deep learning model, iteratively train the preset deep learning model, and verify the preset deep learning model of each iteration based on the validation set to obtain a quality control model to be tested.

[0081] In one or more embodiments of the present specification, the first pathological image and the second pathological image contained in the training set are input as input images into the preset deep learning model, the preset deep learning model is iteratively trained, and the preset deep learning model of each iteration is verified based on the validation set to obtain a quality control model to be tested, which specifically includes the following steps: determining the training set divided based on the data set from the above step S102, and inputting the first pathological image and the second pathological image contained in the training set as input images. In an application scenario of the embodiment of the present specification, the input image pixel size of the quality control network is 384x384, and the pre-training weight of Image Net 22k is used, that is, in order to enhance the image, the image needs to be preprocessed based on the preset mode to obtain the processed input image. For example: in an application scenario of the embodiment of the present specification, the image preprocessing includes upsampling, random rotation, color jittering, dye enhancement, random horizontal, and vertical flipping; specifically, the image is upsampled to 384x384 pixel size, and the image is randomly rotated by 15 0 degrees as the minimum rotation unit, color jittering, dye enhancement, random horizontal or vertical flipping, and finally normalized according to the variance and mean.

[0082] After preprocessing the input image, the processed input image is input into a preset deep learning model, so as to obtain a first prediction result of the preset deep learning model, and the first prediction result is compared with a label value of the input image to obtain a first comparison result. According to the first comparison result and a preset loss function of the preset deep learning model, a first loss function value of each classification layer of the preset deep learning model is determined, and a second loss function value of the preset deep learning model is determined based on the first loss function value. It should be noted that in terms of loss function, the overall loss function value of the quality control model is the sum of the loss function values of each classification layer, which is defined as: where n represents the total number of classification layers, i represents a specific classification layer, and i is the loss function value of the specific classification layer, i.e., the first loss function value, and L is the loss function value of the entire preset deep learning model, i.e., the second loss function value.

[0083] According to the second loss function value and the preset hyperparameters of the preset deep learning model, gradient descent and parameter updating are performed on the preset deep learning model to obtain an updated preset deep learning model. It can be understood that the goal of training the preset deep learning model is to minimize the loss function suitable for the task, and the model parameters can be learned and updated iteratively by the stochastic gradient descent and error backpropagation methods. It should be noted that in one application scenario of the embodiments of the present specification, a cross-entropy loss value function is used, and the preset hyperparameters of the preset deep learning model include a learning rate and an optimization algorithm. For example, the learning rate is 5x10 -6 ; the optimization algorithm is AdamW basic optimization algorithm. In addition, the model uses a SAM optimizer, the weight decay is 0.05, the batch size during training is 32, and the maximum iteration is 50 epochs. After each iteration of the training data set, the updated preset deep learning model needs to be verified based on the verification set, and a third loss function value of the updated preset deep learning model obtained from the verification set is used to end the iterative updating training process of the preset deep learning model, so as to obtain the quality control model to be tested.

[0084] Further, in one or more embodiments of the present specification, the updated preset deep learning model is verified based on the verification set, and the third loss function value of the updated preset deep learning model obtained from the verification set is used to end the iterative updating of the preset deep learning model to obtain the quality control model to be tested. Specifically, the following process is included: first, input the first pathological image and the second pathological image in the verification set into the updated preset deep learning model to obtain the second prediction result of the updated preset deep learning model. Then, the second comparison result is obtained by comparing the second prediction result with the label values of the first pathological image and the second pathological image. The third loss function value of each classification layer of the updated preset deep learning model is determined according to the first comparison result and the preset loss function of the updated preset deep learning model, and the fourth loss function value of the updated preset deep learning model is determined based on the third loss function value. If it is determined that the numerical value of the fourth loss function value increases or remains unchanged, i.e., there is no effective reduction, then the iterative updating process of the preset deep learning model is ended, the updated preset deep learning model is locked, and the updated preset deep learning model is used as the quality control model to be tested. It should be noted that the verification set does not perform image enhancement, gradient descent, and parameter updating.

[0085] S104: Test the quality control model to be tested based on the test set to obtain a quality control model meeting the requirements.

[0086] After the preset deep learning model is trained and verified based on the above step S103, the quality control model to be tested obtained needs to be evaluated. In one or more embodiments of the present specification, the quality control model to be tested is tested by the test set, so as to obtain a quality control model meeting the requirements. It should be noted that in the testing stage, image enhancement, gradient descent, and parameter updating are no longer performed.

[0087] S105: Input the pathological image to be detected into the quality control model meeting the requirements to obtain a quality control result of the pathological image to be detected.

[0088] After the quality control model to be tested is evaluated by the test set, a quality control model meeting the requirements is obtained. By inputting the pathological image to be detected as an input image into the quality control model meeting the requirements, it is determined whether the pathological image to be detected is a pathological image. If it is a pathological image, further identification and classification of the pathological image to be detected are realized based on the fine classification layer of the quality control model, and quality detection of the pathological image to be detected is performed to obtain an effective identification result of the pathological image to be detected, thereby improving the stability and robustness of the slice image analysis algorithm or the computer-aided diagnosis software.

[0089] As Figure 3As shown, in one or more embodiments of the present specification, a deep learning-based pathological image quality control device is provided, and the device comprises:

[0090] A determination module 301 is configured to determine a plurality of classification layers of a preset deep learning model based on preset quality control conditions of a pathological image and business work corresponding to the pathological image.

[0091] A construction module 302 is configured to construct a data set corresponding to the preset deep learning model according to the collected first pathological image without disease and the second pathological image with disease; wherein the data set comprises a training set, a verification set, and a test set.

[0092] A training and verification module 303 is configured to input the first pathological image and the second pathological image contained in the training set as input images into the preset deep learning model, iteratively train the preset deep learning model, and verify the preset deep learning model at each iteration based on the verification set, to obtain a quality control model to be tested.

[0093] A test module 304 is configured to test the quality control model to be tested based on the test set, to obtain a quality control model meeting the requirements.

[0094] An acquisition module 305 is configured to input a to-be-detected pathological image into the quality control model meeting the requirements, to obtain a quality control result of the to-be-detected pathological image.

[0095] Further, in one or more embodiments of the present specification, the device further comprises:

[0096] A selection module is configured to select a network having a corresponding pre-training network on the data set as a backbone network of the preset deep learning model.

[0097] A modification module is configured to modify parameters of classification layers of the backbone network according to preset quality control conditions corresponding to the preset deep learning model, to obtain modified classification layers, and add the classification layers to the backbone network, to obtain new classification layers; wherein the parameters are the number of classification categories.

[0098] A construction module is configured to transfer features of a feature layer located before the classification layers to the modified classification layers and the new classification layers, to construct the preset deep learning model based on the backbone network containing the modified classification layers and the new classification layers.

[0099] Further, in one or more embodiments of the present specification, the determination module specifically comprises:

[0100] a division module configured to divide the preset deep learning model into a main category classification layer according to a business work corresponding to the pathological images;

[0101] a first determination module configured to determine a plurality of fine classification layers of the preset deep learning model according to disease types and attributes of the pathological images corresponding to each main classification layer, and add a preset quality control condition corresponding to each fine classification layer to the fine classification layer;

[0102] a generation module configured to generate a plurality of classification layers of the preset classification model according to the main category classification layer and the fine classification layer.

[0103] Further, in one or more embodiments of the present specification, the construction module specifically includes:

[0104] a collection module configured to collect first pathological images without diseases as a natural image subset and collect first pathological images with diseases as a pathological image subset;

[0105] a constitution module configured to constitute a data set of the preset deep learning model based on the natural image subset and the pathological image subset, and divide the data set into a training set, a tuning set and a test set in proportion.

[0106] Further, in one or more embodiments of the present specification, the training and verification module specifically includes:

[0107] an image selection module configured to take first pathological images and second pathological images contained in the training set as input images;

[0108] a processing module configured to perform image preprocessing on the input images based on a preset manner to obtain processed input images; wherein the preset manner includes up-sampling, random rotation, color jittering, dye enhancement, random horizontal, and vertical flipping;

[0109] a first input module configured to input the processed input images into the preset deep learning model to obtain a first prediction result of the preset deep learning model, and compare the first prediction result with a label value of the input images to obtain a first comparison result;

[0110] a second determination module configured to determine a first loss function value of each classification layer of the preset deep learning model according to the first comparison result and a preset loss function of the preset deep learning model, and determine a second loss function value of the preset deep learning model based on the first loss function value;

[0111] an updating module configured to perform gradient descent and parameter updating on the preset deep learning model according to the second loss function value and preset hyperparameters of the preset deep learning model, to obtain an updated preset deep learning model;

[0112] a verifying module configured to verify the updated preset deep learning model based on the verification set, and end the iterative updating of the preset deep learning model to obtain a quality control model to be tested according to a third loss function value of the updated preset deep learning model obtained based on the verification set.

[0113] Further, in one or more embodiments of the present specification, the verifying module specifically includes:

[0114] a second input module configured to input the first pathological image and the second pathological image in the verification set into the updated preset deep learning model, to obtain a second prediction result of the updated preset deep learning model;

[0115] a comparison module configured to compare the second prediction result with label values of the first pathological image and the second pathological image to obtain a second comparison result;

[0116] a third determination module configured to determine a third loss function value of each classification layer of the updated preset deep learning model according to the first comparison result and a preset loss function of the updated preset deep learning model, and determine a fourth loss function value of the updated preset deep learning model based on the third loss function value;

[0117] an ending module configured to end the iterative updating of the preset deep learning model and take the updated preset deep learning model as the quality control model to be tested if it is determined that the fourth loss function value increases or remains unchanged.

[0118] Further, in one or more embodiments of the present specification, the preset hyperparameters of the preset deep learning model include: a learning rate and an optimization algorithm; wherein the learning rate is: 5x10 -6 ; and the optimization algorithm is: AdamW basic optimization algorithm.

[0119] As shown in Figure 4 one or more embodiments of the present specification, a pathological image quality control device based on deep learning is provided, which includes:

[0120] at least one processor 401; and

[0121] a memory 402 in communication connection with the at least one processor 401; wherein

[0122] The memory 402 stores instructions executable by the at least one processor 401, and the instructions are executed by the at least one processor 401 to enable the at least one processor 401 to:

[0123] Based on the preset quality control condition of the pathological image and the business work corresponding to the pathological image, a plurality of classification layers of a preset deep learning model are determined;

[0124] According to the collected first pathological image without disease and the second pathological image with disease, a data set corresponding to the preset deep learning model is constructed; wherein the data set includes: a training set, a verification set, and a test set;

[0125] The first pathological image and the second pathological image included in the training set are input as input images into the preset deep learning model, the preset deep learning model is iteratively trained, and the preset deep learning model of each iteration is verified based on the verification set, to obtain a quality control model to be tested;

[0126] The quality control model to be tested is tested based on the test set, to obtain a quality control model meeting the requirements;

[0127] The pathological image to be detected is input into the quality control model meeting the requirements, to obtain a quality control result of the pathological image to be detected.

[0128] As shown in Figure 5 The one or more embodiments of the present specification provide a non-volatile storage medium storing computer executable instructions 501, and the computer executable instructions 501 include:

[0129] Based on the preset quality control condition of the pathological image and the business work corresponding to the pathological image, a plurality of classification layers of a preset deep learning model are determined;

[0130] According to the collected first pathological image without disease and the second pathological image with disease, a data set corresponding to the preset deep learning model is constructed; wherein the data set includes: a training set, a verification set, and a test set;

[0131] The first pathological image and the second pathological image included in the training set are input as input images into the preset deep learning model, the preset deep learning model is iteratively trained, and the preset deep learning model of each iteration is verified based on the verification set, to obtain a quality control model to be tested;

[0132] The quality control model to be tested is tested based on the test set, to obtain a quality control model meeting the requirements;

[0133] The pathological image to be detected is input into the qualified quality control model, and a quality control result of the pathological image to be detected is obtained.

[0134] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the device, apparatus, and non-transitory computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0135] The above describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in which they are recited, and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or necessary.

[0136] The above only describes one or more embodiments of the specification, and is not intended to limit the specification. One or more embodiments of the specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of one or more embodiments of the specification shall be included in the scope of the claims of the specification.

Claims

1. A deep learning-based pathological image quality control method, characterized in that, The method comprises: determining a plurality of classification layers of a preset deep learning model based on preset quality control conditions of pathological images and business work corresponding to the pathological images; constructing a data set corresponding to the preset deep learning model according to collected first pathological images without diseases and second pathological images with diseases; wherein the data set comprises a training set, a verification set and a test set; inputting the first pathological images and the second pathological images contained in the training set as input images into the preset deep learning model, iteratively training the preset deep learning model, and verifying the preset deep learning model at each iteration based on the verification set to obtain a quality control model to be tested; testing the quality control model to be tested based on the test set to obtain a quality control model meeting requirements; inputting a pathological image to be detected into the quality control model meeting requirements to obtain a quality control result of the pathological image to be detected; before determining a plurality of classification layers of a preset deep learning model based on preset quality control conditions of pathological images and business work corresponding to the pathological images, the method further comprises: taking a network with a corresponding pre-training network on the data set as a backbone network of the preset deep learning model; modifying parameters of classification layers of the backbone network according to preset quality control conditions corresponding to the preset deep learning model to obtain modified classification layers, and adding the classification layers of the backbone network to obtain new classification layers; wherein the parameters are the number of categories of classification; transferring features of a feature layer located before the classification layers to the modified classification layers and the new classification layers to construct the preset deep learning model based on the backbone network containing the modified classification layers and the new classification layers; determining a plurality of classification layers of a preset deep learning model based on preset quality control conditions of pathological images and business work corresponding to the pathological images, specifically comprising: dividing the preset deep learning model into main category classification layers according to business work corresponding to the pathological images; determining a plurality of fine classification layers of the preset deep learning model according to disease types and attributes of pathological images corresponding to each main classification layer, and adding preset quality control conditions corresponding to each fine classification layer to the fine classification layer; generating a plurality of classification layers of a preset classification model according to the main category classification layers and the fine classification layers.

2. The method of claim 1, wherein the method further comprises: The method of constructing a data set corresponding to the preset deep learning model according to collected first pathological images without diseases and second pathological images with diseases, specifically comprises: collecting first pathological images without diseases as a natural image subset and collecting first pathological images with diseases as a pathological image subset; constructing a data set of the preset deep learning model based on the natural image subset and the pathological image subset, and dividing the data set into a training set, a tuning set and a test set in proportion. 3.The deep learning-based pathological image quality control method of claim 1, wherein, The first pathological image and the second pathological image included in the training set are input as input images into the preset deep learning model, the preset deep learning model is iteratively trained, the preset deep learning model at each iteration is verified based on the verification set, and a quality control model to be tested is obtained, specifically comprising: The first pathological image and the second pathological image included in the training set are input as input images into the preset deep learning model, the preset deep learning model is iteratively trained, the preset deep learning model at each iteration is verified based on the verification set, and a quality control model to be tested is obtained, specifically comprising: The input images are pre-processed based on a preset manner to obtain processed input images; wherein the preset manner includes upsampling, random rotation, color jittering, dye enhancement, random horizontal and vertical flipping; The processed input images are input into the preset deep learning model to obtain a first prediction result of the preset deep learning model, and the first prediction result is compared with the label value of the input image to obtain a first comparison result; According to the first comparison result and a preset loss function of the preset deep learning model, a first loss function value of each classification layer of the preset deep learning model is determined, and a second loss function value of the preset deep learning model is determined based on the first loss function value; According to the second loss function value and the preset hyperparameters of the preset deep learning model, gradient descent and parameter updating are performed on the preset deep learning model to obtain an updated preset deep learning model; The updated preset deep learning model is verified based on the verification set, and a third loss function value of the updated preset deep learning model obtained based on the verification set is used to end the iterative updating of the preset deep learning model to obtain a quality control model to be tested. 4.The deep learning-based pathological image quality control method of claim 3, wherein, The updated preset deep learning model is verified based on the verification set, and a third loss function value of the updated preset deep learning model obtained based on the verification set is used to end the iterative updating of the preset deep learning model to obtain a quality control model to be tested, specifically comprising: The first pathological image and the second pathological image in the verification set are input into the updated preset deep learning model to obtain a second prediction result of the updated preset deep learning model; A second comparison result is obtained by comparing the second prediction result with the label values of the first pathological image and the second pathological image; According to the first comparison result and a preset loss function of the updated preset deep learning model, a third loss function value of each classification layer of the updated preset deep learning model is determined, and a fourth loss function value of the updated preset deep learning model is determined based on the third loss function value; If it is determined that the fourth loss function value increases or remains unchanged, the iterative updating of the preset deep learning model is ended, and the updated preset deep learning model is taken as a quality control model to be tested.

5. The method of claim 3, wherein the method further comprises: The preset hyperparameters of the preset deep learning model include a learning rate and an optimization algorithm. 6.A deep learning-based pathological image quality control device, characterized by, The device comprises: determining a plurality of classification layers of a preset deep learning model based on preset quality control conditions of a pathological image and business work corresponding to the pathological image; a construction module configured to construct a data set corresponding to the preset deep learning model according to the collected first pathological image without disease and the second pathological image with disease; the data set includes a training set, a verification set, and a test set; a training and verification module configured to input the first pathological image and the second pathological image included in the training set as input images into the preset deep learning model, iteratively train the preset deep learning model, and verify the preset deep learning model at each iteration based on the verification set, to obtain a quality control model to be tested; a test module configured to test the quality control model to be tested based on the test set, and obtain a quality control model meeting requirements; an acquisition module configured to input a pathological image to be detected into the quality control model meeting requirements, and obtain a quality control result of the pathological image to be detected; Before determining a plurality of classification layers of a preset deep learning model based on preset quality control conditions of a pathological image and business work corresponding to the pathological image, the method further includes: taking a network with a corresponding pre-training network on the data set as a backbone network of the preset deep learning model; modifying parameters of classification layers of the backbone network according to preset quality control conditions corresponding to the preset deep learning model, obtaining modified classification layers, and adding the classification layers to the backbone network, to obtain new classification layers; the parameters are the number of classification categories; transferring features of a feature layer located before the classification layers to the modified classification layers and the new classification layers, to construct the preset deep learning model based on the backbone network including the modified classification layers and the new classification layers; The method of determining a plurality of classification layers of a preset deep learning model based on preset quality control conditions of a pathological image and business work corresponding to the pathological image specifically includes: dividing the preset deep learning model into main category classification layers according to the business work corresponding to the pathological image; determining a plurality of fine classification layers of the preset deep learning model according to disease types and attributes of pathological images of each main classification layer, and adding preset quality control conditions corresponding to each fine classification layer to the fine classification layer; generating a plurality of classification layers of a preset classification model according to the main category classification layers and the fine classification layers. 7.A deep learning-based pathological image quality control device, characterized by, The device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: determine a plurality of classification layers of a preset deep learning model based on preset quality control conditions of a pathological image and business work corresponding to the pathological image; construct a data set corresponding to the preset deep learning model according to the collected first pathological image without disease and the second pathological image with disease; the data set includes a training set, a verification set, and a test set; Inputting the first pathological image and the second pathological image contained in the training set as input images into the preset deep learning model, iteratively training the preset deep learning model, and verifying the preset deep learning model of each iteration based on the verification set to obtain a quality control model to be tested; Testing the quality control model to be tested based on the test set to obtain a quality control model meeting requirements; Inputting a to-be-detected pathological image into the quality control model meeting requirements to obtain a quality control result of the to-be-detected pathological image; Before determining the multiple classification layers of the preset deep learning model based on the preset quality control condition of the pathological image and the business work corresponding to the pathological image, the method further comprises: Taking a network with a corresponding pre-training network on the data set as a backbone network of the preset deep learning model; According to the preset quality control condition corresponding to the preset deep learning model, modifying parameters of the classification layer of the backbone network to obtain a modified classification layer, and adding the classification layer to the backbone network to obtain an added classification layer; wherein the parameters are the number of classification categories; Transferring features of a feature layer located before the classification layer to the modified classification layer and the added classification layer to construct the preset deep learning model based on the backbone network containing the modified classification layer and the added classification layer; Determining the multiple classification layers of the preset deep learning model based on the preset quality control condition of the pathological image and the business work corresponding to the pathological image, specifically comprising: Dividing the preset deep learning model into main category classification layers according to the business work corresponding to the pathological image; Determining multiple fine classification layers of the preset deep learning model according to the disease types and attributes of the pathological images of each main classification layer, and adding the preset quality control condition corresponding to each fine classification layer to the fine classification layer; Generating the multiple classification layers of the preset classification model according to the main category classification layers and the fine classification layers.

8. A non-transitory storage medium storing computer-executable instructions, the computer-executable instructions comprising: The computer executable instructions comprise: Determining the multiple classification layers of the preset deep learning model based on the preset quality control condition of the pathological image and the business work corresponding to the pathological image; Constructing a data set corresponding to the preset deep learning model based on the collected first pathological image without disease and the second pathological image with disease; wherein the data set comprises: a training set, a verification set, and a test set; Inputting the first pathological image and the second pathological image contained in the training set as input images into the preset deep learning model, iteratively training the preset deep learning model, and verifying the preset deep learning model of each iteration based on the verification set to obtain a quality control model to be tested; Testing the quality control model to be tested based on the test set to obtain a quality control model meeting requirements; Inputting a to-be-detected pathological image into the quality control model meeting requirements to obtain a quality control result of the to-be-detected pathological image; Before determining the multiple classification layers of the preset deep learning model based on the preset quality control condition of the pathological image and the business work corresponding to the pathological image, the method further comprises: a network with corresponding pre-training network on the dataset is taken as a backbone network of the preset deep learning model; parameters of a classification layer of the backbone network are modified according to preset quality control conditions corresponding to the preset deep learning model, to obtain a modified classification layer, and the classification layer of the backbone network is added to obtain an added classification layer; wherein the parameters are the number of classification categories; features of a feature layer located before the classification layer are transmitted to the modified classification layer and the added classification layer, to construct the preset deep learning model based on the backbone network containing the modified classification layer and the added classification layer; a plurality of classification layers of a preset deep learning model are determined based on preset quality control conditions of a pathological image and business work corresponding to the pathological image, specifically including: the preset deep learning model is divided into main category classification layers according to the business work corresponding to the pathological image; a plurality of fine classification layers of the preset deep learning model are determined according to disease types and attributes of pathological images of each main classification layer, and preset quality control conditions corresponding to each fine classification layer are added to the fine classification layer; a plurality of classification layers of a preset classification model are generated according to the main category classification layer and the fine classification layer.

Images