A method, apparatus, device and storage medium for classifying pathological images

Abstract

The application relates to a classification method, device and equipment for pathological images and a storage medium. The method comprises the following steps: processing a full-view digital pathological image to be detected to obtain a down-sampling image block; constructing a distillation and adversarial network, training the distillation and adversarial network, and extracting a classification sub-network from the trained distillation and adversarial network; inputting the down-sampling image block into the classification sub-network for classification to obtain an initial classification result; and obtaining a classification result corresponding to the full-view digital pathological image to be detected based on the initial classification result. According to the method, small data amount down-sampling image blocks can be input into a light classification sub-network to quickly and efficiently classify corresponding pathological images.

Description

Technical Field

[0001] This application relates to the fields of medical image processing technology and artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for classifying pathological images. Background Technology

[0002] Pathological images are the gold standard for definitive cancer diagnosis. Methods relying on doctors manually classifying pathological images are not only time-consuming and labor-intensive, but their diagnostic results are also easily influenced by subjective human factors such as the doctor's experience and skill level.

[0003] Using artificial intelligence (AI) to empower pathological image analysis presents a good technological opportunity for the close integration of AI technology with medical knowledge. Deep learning-based pathological image classification methods often employ fully supervised learning, feeding labeled pathological images of lesion regions into a neural network for training.

[0004] However, because pathological images are typically ultra-high resolution, a single pathological image can easily exceed 300MB, and even reach 5GB in size. Such a massive amount of data leads to drawbacks when using current deep learning models to classify pathological images, including long training times, slow prediction speeds, and overall time consumption. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, device, and storage medium for classifying pathological images that can quickly and efficiently classify pathological images, addressing the aforementioned technical problems.

[0006] Firstly, this application provides a classification method for pathological images. The method includes:

[0007] The full-view digital pathology image to be tested is processed to obtain downsampled image blocks;

[0008] Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network;

[0009] The downsampled image patch is imported into the classification subnetwork for classification to obtain an initial classification result;

[0010] Based on the initial classification results, the classification results corresponding to the full-view digital pathological image to be detected are obtained.

[0011] In one embodiment, the process of processing the full-view digital pathological image to be detected to obtain downsampled image blocks includes:

[0012] Downsampling is performed on the full-view digital pathology images to be tested;

[0013] The downsampled full-view digital pathological image to be detected is preprocessed to obtain downsampled image blocks.

[0014] In one embodiment, the preprocessing of the downsampled full-view digital pathological image to be detected to obtain a downsampled image block includes:

[0015] The downsampled full-view digital pathological image to be detected is filtered and segmented to obtain downsampled image blocks.

[0016] In one embodiment, the construction of the distillation and adversarial network, training the distillation and adversarial network, and extracting a classification subnetwork from the trained distillation and adversarial network include:

[0017] A distillation and adversarial network is established, which includes a teacher classification network, a generator, a discriminator, and a student classification network;

[0018] Build the training dataset;

[0019] The training dataset is imported into the distillation and adversarial network for training, resulting in the trained distillation and adversarial network.

[0020] The generator and the student classification network are extracted from the trained distillation and adversarial network, and a classification subnetwork is constructed based on the generator and the student classification network.

[0021] In one embodiment, constructing the training dataset includes:

[0022] Collect high-resolution, full-view digital pathology images;

[0023] The high-resolution full-view digital pathology image is downsampled to obtain a low-resolution full-view digital pathology image.

[0024] The high-resolution full-view digital pathology image and the low-resolution full-view digital pathology image are preprocessed to obtain high-resolution image blocks and low-resolution image blocks.

[0025] A training dataset is constructed based on the high-resolution image patches and the low-resolution image patches.

[0026] In one embodiment, the step of importing the training dataset into the distillation and adversarial network for training to obtain the trained distillation and adversarial network includes:

[0027] The training dataset includes high-resolution image patches and low-resolution image patches;

[0028] The high-resolution image patch is used as input to the teacher classification network to train the teacher classification network;

[0029] The low-resolution image patch is input into the generator to obtain a newly generated image patch. The newly generated image patch and the high-resolution image patch are then input into the discriminator to train the discriminator.

[0030] Based on the trained teacher classification network and the discriminator, the high-resolution image patch and the low-resolution image patch are simultaneously used as inputs to the distillation and adversarial network to train the generator and the student classification network.

[0031] Based on the trained teacher classification network, generator, discriminator, and student classification network, the trained distillation and adversarial network is obtained.

[0032] In one embodiment, obtaining the classification result corresponding to the full-view digital pathology image to be detected based on the initial classification result includes:

[0033] Based on the number of downsampled image patches of different categories in the initial classification results, the category with the larger number of downsampled image patches is taken as the classification result of the full-view digital pathology image to be detected.

[0034] Secondly, this application also provides a classification device for pathological images. The device includes:

[0035] The processing module is used to process the full-view digital pathological image to be detected to obtain downsampled image blocks;

[0036] A network module is used to construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network.

[0037] The first classification module is used to import the downsampled image block into the classification sub-network for classification to obtain an initial classification result;

[0038] The second classification module obtains the classification result corresponding to the full-view digital pathological image to be detected based on the initial classification result.

[0039] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0040] The full-view digital pathology image to be tested is processed to obtain downsampled image blocks;

[0041] Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network;

[0042] The downsampled image patch is imported into the classification subnetwork for classification to obtain an initial classification result;

[0043] Based on the initial classification results, the classification results corresponding to the full-view digital pathological image to be detected are obtained.

[0044] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0045] The full-view digital pathology image to be tested is processed to obtain downsampled image blocks;

[0046] Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network;

[0047] The downsampled image patch is imported into the classification subnetwork for classification to obtain an initial classification result;

[0048] Based on the initial classification results, the classification results corresponding to the full-view digital pathological image to be detected are obtained.

[0049] The aforementioned classification method, apparatus, device, and storage medium for pathological images reduce the amount of data input to the classification network by using low-resolution downsampled image patches as input samples, thereby reducing the prediction time of classification results. Furthermore, by constructing a distillation and adversarial network capable of accurately classifying pathological images and extracting a lightweight classification sub-network from it as the final classification network for disease classification, the device's memory consumption is reduced, the prediction speed of classification results is accelerated, and the efficiency of pathological image classification is significantly improved. Attached Figure Description

[0050] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0051] Figure 1 This is an application environment diagram of a classification method for pathological images in one embodiment;

[0052] Figure 2 This is a flowchart illustrating a classification method for pathological images in one embodiment;

[0053] Figure 3This is a schematic diagram of the process of training the discriminator in one embodiment;

[0054] Figure 4 This is a flowchart illustrating the training distillation and adversarial network process in one embodiment;

[0055] Figure 5 This is a flowchart illustrating a preferred embodiment of a method for classifying pathological images.

[0056] Figure 6 This is a structural block diagram of a classification device for pathological images in one embodiment;

[0057] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0059] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0060] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the pathological image classification method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0061] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the classification method for pathological images in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the methods described above. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0062] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0063] This embodiment provides a classification method for pathological images. Figure 2 This is a flowchart of the classification method for pathological images in this embodiment, as shown below. Figure 2 As shown, the process includes the following steps:

[0064] Step S210: Process the full-view digital pathological image to be detected to obtain downsampled image blocks.

[0065] Full-field digital pathology images utilize fully automated microscope scanners and computer technology to visualize pathological slides as high-resolution digital images. The full-field digital pathology images undergo downsampling to reduce their resolution and data volume. Then, the downsampled images are automatically segmented, filtering out blank backgrounds and natural holes, and cut into multiple small image blocks of size n×n, where n represents the length and width of each small image block. These small image blocks are the downsampled image blocks.

[0066] Step S220: Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network.

[0067] Among them, the distillation and adversarial network is a complex network based on contrastive representation learning and conditional adversarial methods, including a teacher classification network C. T Student classification network C S Generator G and discriminator D. The generator G and student classification network C are extracted from the trained distillation and adversarial network. S As a classification subnetwork, the classification subnetwork is a simple and lightweight classification network. Based on the training of distillation and adversarial networks, the classification subnetwork has a faster classification speed while ensuring the classification accuracy of pathological images.

[0068] Step S230: The downsampled image patch is imported into the classification sub-network for classification to obtain the initial classification result. The initial classification result represents the category of the disease corresponding to the full-view digital pathology image.

[0069] Step S240: Based on the initial classification results, obtain the classification results of the corresponding full-view digital pathology image to be detected. By comprehensively analyzing the initial classification results, classify the full-view digital pathology image and predict its disease category.

[0070] Compared with existing technologies, the classification method for pathological images provided in this embodiment reduces the amount of data input to the classification network and the time required to predict classification results by using low-resolution downsampled image patches as input samples. On the other hand, it constructs a distillation and adversarial network that can accurately classify pathological images and extracts a lightweight classification sub-network from it as the final classification network for disease classification, thereby reducing the memory consumption of the device, accelerating the prediction speed of classification results, and significantly improving the efficiency of pathological image classification.

[0071] Furthermore, in one embodiment, based on the above step S201, the full-view digital pathological image to be detected is processed to obtain downsampled image blocks, which may specifically include the following steps:

[0072] Step S211: Downsampling is performed on the full-view digital pathology image to be detected.

[0073] Specifically, in each row and column of the original full-view digital pathology image, a new image is formed by taking one point every k points to create a new image.

[0074] Step S212: Preprocess the downsampled full-view digital pathological image to be detected to obtain downsampled image blocks.

[0075] Furthermore, in one embodiment, based on the above step S212, the following steps are specifically included:

[0076] The downsampled full-view digital pathology image to be tested is filtered and segmented to obtain downsampled image blocks. For example, the filtering process specifically involves edge detection and hole detection on the downsampled full-view digital pathology image to obtain the tissue regions, and removing background regions and natural holes outside the tissue regions. The filtered full-view digital pathology image is then segmented into multiple small image blocks of size 256×256, each of which is a downsampled image block.

[0077] Steps S211 to S212 above involve downsampling, filtering, and segmenting the full-view digital pathological image to be detected, which reduces the resolution and removes irrelevant pixels, resulting in a downsampled image block with a lower data volume.

[0078] Furthermore, in one embodiment, based on the above step S220, a distillation and adversarial network is constructed, the distillation and adversarial network is trained, and a classification sub-network is extracted from the trained distillation and adversarial network. Specifically, this may include the following steps:

[0079] Step S221: Establish the distillation and adversarial network, which includes a teacher classification network C. T Generator G, discriminator D, and student classification network C S .

[0080] Among them, the teacher classification network C T To distill and transfer learned classification knowledge to the student classification network C S In the middle, as the preferred teacher classification network C TResNet-50, ResNet-101, or ResNet-152 can be used. The generator G takes low-resolution image data as input and outputs new image data. Under the guidance of the discriminator, it tries to fit the new output image data as closely as possible to the high-resolution image data. The discriminator D is used to distinguish between real high-resolution image data and the new image data generated by generator G, in order to optimize generator G. Generator G can be any conventional generator G; for example, generator G consists of four residual blocks and three small-stride convolutional layers. Student classification network C... S Used to learn teacher classification network C T Distillation transfer classification knowledge is a classification network with lower latency and fewer parameters, used to make the final judgment on the input image data, serving as the preferred student classification network C. S ResNet-18 can be used.

[0081] Step S222: Construct a training dataset. Different training datasets need to be constructed for the diagnosis of cancer subtypes in different organs. For example, to construct a training dataset for classifying lung adenocarcinoma and lung squamous cell carcinoma in lung cancer, full-view digital pathological images of lung adenocarcinoma and lung squamous cell carcinoma need to be collected.

[0082] Furthermore, in one embodiment, constructing a training dataset based on the above step S222 may specifically include the following steps:

[0083] Step S2221: Collect high-resolution full-view digital pathology images.

[0084] Three thousand full-field digital pathology images of lung cancer were collected from the open-source databases TCGA and TCIA, including 1,516 images of lung adenocarcinoma and 1,484 images of lung squamous cell carcinoma, maintaining a uniform distribution of data between the two lung cancer subtypes. The images were labeled to obtain the ground truth labels for the full-field digital pathology images, with lung adenocarcinoma labeled as 0 and lung squamous cell carcinoma labeled as 1. All data contained no pixel-level, patch-level, or ROI-level labels.

[0085] Therefore, training the distillation and adversarial network requires no pixel-level, block-level, or ROI-level annotations; only disease subtype labels from full-view digital pathology images are needed, greatly reducing the workload of dataset annotation. Furthermore, the required full-view digital pathology image data is not limited by disease type and can be applied to cancer diagnosis tasks in various organs, demonstrating its universality.

[0086] Step S2222: Downsampling is performed on the high-resolution full-view digital pathology image to obtain a low-resolution full-view digital pathology image.

[0087] In another embodiment, the following steps are included before step S2222 described above:

[0088] Color normalization is performed on the original high-resolution full-view digital pathology images to eliminate the negative impact of staining differences in the pathology images on the network. For example, a high-resolution full-view digital pathology image with clear and standard staining is selected as the template image. The Sparse Stain Separation Color Normalization (SPCN) algorithm is used to normalize the colors of the remaining high-resolution full-view digital pathology images, so that the color distribution of all high-resolution full-view digital pathology images is the same.

[0089] Step S2223: Preprocess the high-resolution full-view digital pathology image and the low-resolution full-view digital pathology image to obtain high-resolution image patch m. high and low-resolution image patches m low .

[0090] The preprocessing includes automatically segmenting high-resolution full-view digital pathology images and low-resolution full-view digital pathology images, filtering out blank backgrounds and natural holes, and cutting them into several small image blocks of 256×256.

[0091] Step S2224, based on high-resolution image patch m high and low-resolution image patches m low Construct the training dataset.

[0092] Among them, high-resolution image block m high C for training teacher classification networks T To obtain a teacher classification network C with high classification accuracy. T High-resolution image patch m high and low-resolution image patches m low Together they act on the student classification network C S A generator G and a discriminator D are used to train a lightweight student classification network C. S .

[0093] Step S223: Import the training dataset into the distillation and adversarial network for training to obtain the trained distillation and adversarial network.

[0094] Furthermore, in one embodiment, step S223 described above may specifically include the following steps:

[0095] Step S2231, the training dataset includes high-resolution image patches m high and low-resolution image patches m low The method for constructing the training dataset is as described in steps S2221 to S2224 above, and will not be repeated here.

[0096] Step S2232, divide the high-resolution image block mhigh As a teacher classification network C T Input, training teacher classification network C T .

[0097] In this method, high-resolution pathological image patches are used as input, and stochastic gradient descent is employed for backpropagation of gradients. The goal is to train and optimize the teacher classification network C by minimizing the cross-entropy loss function. T The network parameters were set using the Adam optimizer with an initial learning rate of 0.0001. Early stopping was used to update and save the parameters in a timely manner. The trained teacher classification network C... T This will be used as a diagnostic reference to assist in training the student classification network C. S .

[0098] Step S2233, divide the low-resolution image block m low Input generator G to obtain newly generated image patch m low，fake The newly generated image patch m low，fake and high-resolution image blocks m high Input them together into the discriminator D to train the discriminator D.

[0099] Specifically, the network parameters of the generator G and discriminator D are randomly initialized, the network parameters of the generator G are fixed, and only the discriminator D is trained, such as... Figure 3 As shown, the specific process is as follows:

[0100] low-resolution image patch m 1ow The image is fed into generator G to obtain a newly generated image patch m. 1ow，fake The formula is as follows:

[0101] m low，fake =G(m) low );

[0102] The newly generated image patch m low，fake and high-resolution image blocks m high The data is fed into the discriminator D to calculate the first discriminant loss L. D1 Use stochastic gradient descent to maximize the first discriminative loss L. D1 The discriminator D is trained to optimize and update its parameters. This process is repeated 5 times to obtain the trained discriminator D.

[0103] Step S2234, based on the trained teacher classification network C T And discriminator D, to classify high-resolution image patch m high and low-resolution image patches m low Simultaneously, the generator G and the student classification network C serve as inputs to the distillation and adversarial networks. S .

[0104] Specifically, the random initialization generator G and the student classification network C S The network parameters of the fixed-train discriminator D are given, and the generator G and student classification network C are trained. S ,like Figure 4 As shown, the specific process is as follows:

[0105] low-resolution image patch m 1ow The image is fed into generator G to obtain a newly generated image patch m. 1ow，fake The formula is as follows:

[0106] m low，fake =G(m) low );

[0107] The newly generated image patch m 1ow，fake and high-resolution image blocks m high The data is fed into the trained discriminator D to calculate the second discriminant loss L. D ; The newly generated image patch m low，fake Send to network C S In the middle, through the student classification network C S The classification loss L is calculated by combining the output results with the real labels. C And through the student classification network C s The output results are combined with the teacher classification network C T The output soft label calculates the distillation loss L. T The classification loss L C Using the loss function, distillation loss L T Use KL divergence; where the soft label is the high-resolution image patch m. high Input the trained teacher classification network C T Subsequently, the teacher classification network C T The output classification results.

[0108] Calculate the total loss function L total The total loss function includes the discriminant loss L. D Classification loss L C and distillation loss L T The specific formula is as follows:

[0109] L total =λ D L D +λ C L C +λ T L T

[0110] Where each λ represents a weight of the corresponding loss function. For example, let λ be set to... T =10, λ D =5,λC =5.

[0111] Use the stochastic gradient descent algorithm to minimize the total loss function L. total Training is performed with Adam as the optimizer, updating and optimizing the first-order generator G and the student classification network C. S Network parameters.

[0112] Step S2235, based on the trained teacher classification network C T Generator G, discriminator D, and student classification network C S The trained distillation and adversarial network are obtained.

[0113] Steps S2231 to S2235 above are based on the teacher classification network C. T The output is a soft label, guiding students to classify the network C. S The optimization direction, relative to the student classification network C S Optimization learning directly from raw data results in faster learning speed and better performance. Furthermore, soft labels, through label smoothing training, enable the student classification network C. S Provide regularization constraints to avoid student classification network C S This approach relies heavily on the true labels of training samples to prevent overfitting and improve the model's robustness. A complex teacher classification network, C, is implemented at the cost of slight performance loss. T Knowledge transfer to a simple, lightweight student classification network C S In this study, without reducing classification accuracy, a significant improvement in pathological image prediction speed is achieved by using downsampled low-resolution pathological images as input.

[0114] Furthermore, in one embodiment, prior to step S2234 described above, the following step is also included:

[0115] Steps S2233 to S2234 are repeated sequentially to iteratively optimize the discriminator D, generator G, and student classification network C. S The network parameters are determined, and training stops when the total loss function value no longer decreases after 30 consecutive iterations. Ten-fold cross-validation is used to determine the optimal model, resulting in the trained distillation and adversarial network.

[0116] Step S224: Extract the generator G and student classification network C from the trained distillation and adversarial network. S Based on generator G and student classification network C S Construct a classification subnetwork.

[0117] Steps S221 to S224 above design a distillation and adversarial network. The network optimization process is a binary minimax game, where the network parameters of one side are fixed during training, and then the network parameters of the other side are updated, alternating iteratively. Using low-resolution image patches m... low As input, the generator G simulates and generates image patches m with high resolution as much as possible. high Fake images with feature information are used as student classification network C S The input is optimized to preserve as much information as possible from the original high-resolution pathological images, thereby improving the model's accuracy. Furthermore, a classification loss L is introduced. C Using conditional variables to guide data towards trends that favor accurate diagnosis of cancer subtypes reduces the degrees of freedom of the generative adversarial network, further improving classification performance.

[0118] Furthermore, in one embodiment, based on the above step S240, obtaining the classification result of the corresponding full-view digital pathological image to be detected based on the initial classification result may specifically include the following steps:

[0119] Based on the number of downsampled image patches of different categories in the initial classification results, the category with the most downsampled image patches is taken as the classification result of the full-view digital pathology image to be detected. Taking the classification of the above-mentioned lung cancer subtypes as an example, the number of downsampled image patches classified as lung adenocarcinoma and the number of downsampled image patches classified as lung squamous cell carcinoma in the initial classification results are counted. A majority voting method is used to determine the classification result of the full-view digital pathology image to be detected based on the number of lung cancer subtypes with the largest number, thereby improving the accuracy of the classification results.

[0120] The present embodiment will now be described and illustrated through preferred embodiments.

[0121] Figure 5 This is a flowchart of a preferred embodiment of a method for classifying pathological images.

[0122] Step S501: Downsampling is performed on the full-view digital pathology image to be detected.

[0123] Step S502: Preprocess the downsampled full-view digital pathological image to be detected to obtain downsampled image blocks.

[0124] Step S503: Establish the distillation and adversarial network, which includes a teacher classification network C. T Generator G, discriminator D, and student classification network C S .

[0125] Step S504: Construct the training dataset.

[0126] Step S505: Import the training dataset into the distillation and adversarial network for training to obtain the trained distillation and adversarial network.

[0127] Step S506: Extract the generator G and student classification network C from the trained distillation and adversarial network. S Based on generator G and student classification network C S Construct a classification subnetwork.

[0128] Step S507: Import the downsampled image patch into the classification subnetwork for classification to obtain the initial classification result.

[0129] Step S508: Based on the number of downsampled image patches of different categories in the initial classification results, the category with the most downsampled image patches is taken as the classification result of the full-view digital pathology image to be detected.

[0130] This embodiment also provides a classification device for pathological images, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0131] Figure 6 This is a structural block diagram of the classification device 60 for pathological images in this embodiment, as shown below. Figure 6 As shown, the device includes a processing module 61, a network module 62, a first classification module 63, and a second classification module 64.

[0132] The processing module 61 is used to process the full-view digital pathological image to be detected to obtain downsampled image blocks.

[0133] Full-field digital pathology images utilize fully automated microscope scanners and computer technology to visualize pathological slides as high-resolution digital images. The full-field digital pathology images undergo downsampling to reduce their resolution and data volume. Then, the downsampled images are automatically segmented, filtering out blank backgrounds and natural holes, and cut into multiple small image blocks of size n×n, where n represents the length and width of each small image block. These small image blocks are the downsampled image blocks.

[0134] Network module 62 is used to construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network.

[0135] Among them, the distillation and adversarial network is a complex network based on contrastive representation learning and conditional adversarial methods, including a teacher classification network C. T Student classification network C S Generator G and discriminator D. The generator G and student classification network C are extracted from the trained distillation and adversarial network. S As a classification subnetwork, the classification subnetwork is a simple and lightweight classification network. Based on the training of distillation and adversarial networks, the classification subnetwork has a faster classification speed while ensuring the classification accuracy of pathological images.

[0136] The first classification module 63 is used to import downsampled image blocks into the classification sub-network for classification to obtain initial classification results. These initial classification results represent the category of the disease corresponding to the full-view digital pathology image.

[0137] The second classification module 64 obtains the classification results of the corresponding full-view digital pathological image to be detected based on the initial classification results.

[0138] Compared with existing technologies, the classification method for pathological images provided in this embodiment reduces the amount of data input to the classification network and the time required to predict classification results by using low-resolution downsampled image patches as input samples. On the other hand, it constructs a distillation and adversarial network that can accurately classify pathological images and extracts a lightweight classification sub-network from it as the final classification network for disease classification, thereby reducing the memory consumption of the device, accelerating the prediction speed of classification results, and significantly improving the efficiency of pathological image classification.

[0139] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0140] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a classification method for pathological images. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0141] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0143] Step S210: Process the full-view digital pathology image to be detected to obtain downsampled image blocks;

[0144] Step S220: Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network.

[0145] Step S230: Import the downsampled image patch into the classification subnetwork for classification to obtain the initial classification result;

[0146] Step S240: Based on the initial classification results, obtain the classification results of the corresponding full-view digital pathological image to be detected.

[0147] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0148] Step S210: Process the full-view digital pathology image to be detected to obtain downsampled image blocks;

[0149] Step S220: Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network.

[0150] Step S230: Import the downsampled image patch into the classification subnetwork for classification to obtain the initial classification result;

[0151] Step S240: Based on the initial classification results, obtain the classification results of the corresponding full-view digital pathological image to be detected.

[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0153] Step S210: Process the full-view digital pathology image to be detected to obtain downsampled image blocks;

[0154] Step S220: Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network.

[0155] Step S230: Import the downsampled image patch into the classification subnetwork for classification to obtain the initial classification result;

[0156] Step S240: Based on the initial classification results, obtain the classification results of the corresponding full-view digital pathological image to be detected.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0158] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A classification method for pathological images, characterized in that, The method includes: The full-view digital pathology image to be tested is processed to obtain downsampled image blocks; Construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network; The downsampled image patch is imported into the classification subnetwork for classification to obtain an initial classification result; Based on the initial classification results, the classification results corresponding to the full-view digital pathology image to be detected are obtained; The construction of the distillation and adversarial network, training the distillation and adversarial network, and extracting a classification subnetwork from the trained distillation and adversarial network include: A distillation and adversarial network is established, which includes a teacher classification network, a generator, a discriminator, and a student classification network; Construct a training dataset; import the training dataset into the distillation and adversarial network for training, resulting in the trained distillation and adversarial network. Extract the generator and the student classification network from the trained distillation and adversarial network, and construct a classification subnetwork based on the generator and the student classification network; The teacher classification network is used to transfer the learned classification knowledge to the student classification network. The generator takes low-resolution image data as input, outputs new image data, and fits high-resolution image data based on the new output image data under the guidance of the discriminator. The discriminator is used to distinguish between real high-resolution image data and new image data generated by the generator in order to optimize the generator. The student classification network is used to classify based on the high-resolution image data provided by the generator.

2. The classification method for pathological images according to claim 1, characterized in that, The process of processing the full-view digital pathological image to be detected to obtain downsampled image blocks includes: Downsampling is performed on the full-view digital pathology images to be tested; The downsampled full-view digital pathological image to be detected is preprocessed to obtain downsampled image blocks.

3. The classification method for pathological images according to claim 2, characterized in that, The preprocessing of the downsampled full-view digital pathological image to be detected to obtain downsampled image blocks includes: The downsampled full-view digital pathological image to be detected is filtered and segmented to obtain downsampled image blocks.

4. The classification method for pathological images according to claim 1, characterized in that, The construction of the training dataset includes: Collect high-resolution, full-view digital pathology images; The high-resolution full-view digital pathology image is downsampled to obtain a low-resolution full-view digital pathology image. The high-resolution full-view digital pathology image and the low-resolution full-view digital pathology image are preprocessed to obtain high-resolution image blocks and low-resolution image blocks. A training dataset is constructed based on the high-resolution image patches and the low-resolution image patches.

5. The classification method for pathological images according to claim 1, characterized in that, The step of importing the training dataset into the distillation and adversarial network for training, to obtain the trained distillation and adversarial network, includes: The training dataset includes high-resolution image patches and low-resolution image patches; The high-resolution image patch is used as input to the teacher classification network to train the teacher classification network; The low-resolution image patch is input into the generator to obtain a newly generated image patch. The newly generated image patch and the high-resolution image patch are then input into the discriminator to train the discriminator. Based on the trained teacher classification network and the discriminator, the high-resolution image patch and the low-resolution image patch are simultaneously used as inputs to the distillation and adversarial network to train the generator and the student classification network. Based on the trained teacher classification network, generator, discriminator, and student classification network, the trained distillation and adversarial network is obtained.

6. The classification method for pathological images according to claim 1, characterized in that, The process of obtaining the classification result corresponding to the full-view digital pathology image to be detected based on the initial classification result includes: Based on the number of downsampled image patches of different categories in the initial classification results, the category with the larger number of downsampled image patches is taken as the classification result of the full-view digital pathology image to be detected.

7. A classification device for pathological images, characterized in that, The device includes: The processing module is used to process the full-view digital pathological image to be detected to obtain downsampled image blocks; A network module is used to construct a distillation and adversarial network, train the distillation and adversarial network, and extract a classification subnetwork from the trained distillation and adversarial network. The module includes: establishing a distillation and adversarial network, which includes a teacher classification network, a generator, a discriminator, and a student classification network; constructing a training dataset; importing the training dataset into the distillation and adversarial network for training to obtain a trained distillation and adversarial network; extracting the generator and the student classification network from the trained distillation and adversarial network; and constructing a classification subnetwork based on the generator and the student classification network. The teacher classification network is used to transfer the learned classification knowledge to the student classification network. The generator takes low-resolution image data as input, outputs new image data, and fits high-resolution image data based on the new output image data under the guidance of the discriminator. The discriminator is used to distinguish between real high-resolution image data and new image data generated by the generator in order to optimize the generator. The student classification network is used to classify based on the high-resolution image data provided by the generator. The first classification module is used to import the downsampled image block into the classification sub-network for classification to obtain an initial classification result; The second classification module obtains the classification result corresponding to the full-view digital pathological image to be detected based on the initial classification result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the classification method for pathological images according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the classification method for pathological images as described in any one of claims 1 to 6.

Images