Interpretable cancer pathology slice image recognition method, device and computer equipment

Abstract

The present invention belongs to the field of medical image processing and relates to an interpretable cancer pathology slice image recognition method, apparatus, and computer equipment. The method comprises acquiring a high-resolution medical image of a cancer pathology slice and processing it into image blocks; inputting the image blocks into a feature extraction network to extract feature vectors; inputting the feature vectors of the first image block into an image recognition network for pre-training; inputting the feature vectors of the second image block into the pre-trained image recognition network to output a first interpretable heat map; optimizing using a sparse representation method and judging the optimized first interpretable heat map; retraining the image recognition network based on the judgment results; inputting the feature vectors of the third image block into the retrained image recognition network to output a second interpretable heat map. The present invention enables high-resolution images to be predicted in a model and provide professional, interpretable results, achieving a two-way interaction between data and medical domain knowledge.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and in particular relates to an interpretable cancer pathology slice image recognition method, device and computer equipment. Background Art

[0002] With the development of whole-slide scanning technology, a large number of tissue sections are scanned as full-field digital pathology sections (Wholeslide images, WSIs) and stored in digital form. By using artificial intelligence (AI) methods such as machine learning algorithms and deep learning algorithms, computers are applied to pathology section analysis. With the help of computer computing power, learning can be performed on a large number of WSIs, and the trained models can be used to analyze the WSIs. The results can be visualized to assist doctors in diagnosis, thereby making full use of the rich image information contained in the WSIs.

[0003] Currently, in the process of using AI methods to analyze pathological sections, the high-resolution cancer tumor sections obtained by scanning are mainly used as data sets to train the model in a weakly supervised manner. Finally, the model can predict and score the tumor type of a case section, and generate an attention score for each section. Based on the score, the most likely source location of the tumor can be determined.

[0004] However, in AI healthcare, a qualified analysis and recognition system must be transparent, explainable, and understandable to gain the trust of patients and physicians. The interpretability of an analysis and recognition system corresponds to the interpretability of the corresponding neural network model, and interpretation requires the use of a language to describe and annotate it. In practice, neural network models often make prediction errors, but the source of these errors is unclear. Explaining these errors is necessary to help address them and improve the model. Furthermore, applying end-to-end deep learning models to infer cancer location and type from medical cancer biopsies to clinical diagnosis has limitations, largely stemming from the interpretability of black-box models. Summary of the Invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the present invention proposes an interpretable cancer pathology section image recognition method, device and computer equipment, and provides a pathology section cancer interpretable discrimination model based on human-computer interactive learning, improves the traditional model, and provides a new model interpretable framework. The improved model interpretable framework includes four parts: p1, p2, p3 and p4. P1 is for processing high-resolution medical images, p2 is for the training process of high-resolution medical images, p3 is for interpretable learning based on human-computer interactive learning, and p4 is for sparse learning of interpretable results. The four parts are used to optimize the cancer pathology section image recognition results and interpretable results.

[0006] In a first aspect of the present invention, the present invention provides an interpretable cancer pathology slide image recognition method, the method comprising:

[0007] Acquire a high-resolution medical image of a cancer pathology section, pre-process and divide it into image blocks of uniform size; and process the high-resolution medical images from the prediction training set, the explanation training set, and the test set into a first image block, a second image block, and a third image block, respectively;

[0008] Inputting the image block into a pre-trained feature extraction network to extract a feature vector of each image block;

[0009] Inputting the feature vector of the first image block into a preset image recognition network to pre-train the preset image recognition network;

[0010] Inputting the feature vector of the second image block into the pre-trained image recognition network and outputting a first interpretable heat map;

[0011] Optimizing the first interpretable heat map using a sparse representation method, and judging the optimized first interpretable heat map;

[0012] replacing the erroneous portion of the first interpretable heat map determined to be erroneous with a counterexample, and retraining the image recognition network;

[0013] Alternatively, providing a mask for an erroneous portion of the first interpretable heat map that was incorrectly determined, correcting the erroneous interpretable heat map using a second loss function, and retraining the image recognition network;

[0014] The feature vector of the third image patch is input into the retrained image recognition network, and a second interpretable heat map is output.

[0015] In a second aspect of the present invention, the present invention further provides an interpretable cancer pathology slice image recognition device, which is used to implement the interpretable cancer pathology slice image recognition method as described in the first aspect of the present invention, and the device comprises:

[0016] An image acquisition module for acquiring high-resolution medical images of cancer pathology sections;

[0017] An image processing module, configured to pre-process the acquired high-resolution medical image of the cancer pathology section and divide the image into image blocks of uniform size;

[0018] A feature extraction module, configured to input the image blocks into a pre-trained feature extraction network to extract a feature vector for each image block;

[0019] a pre-training module, configured to input the feature vector of the first image block into a preset image recognition network to pre-train the preset image recognition network;

[0020] an image optimization module, configured to input the feature vector of the second image block into a pre-trained image recognition network, output a first interpretable heat map, optimize the first interpretable heat map using a sparse representation method, and perform judgment on the optimized first interpretable heat map;

[0021] a retraining module, configured to replace the erroneous portion of the first interpretable heat map determined to be erroneous with a counterexample, and retrain the image recognition network; or, to provide a mask to the erroneous portion of the first interpretable heat map determined to be erroneous, correct the erroneous interpretable heat map using a second loss function, and retrain the image recognition network;

[0022] The image recognition module is configured to input the feature vector of the third image block into the retrained image recognition network and output a second interpretable heat map.

[0023] In the third aspect of the present invention, the present invention also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the identification method described in the first aspect of the present invention.

[0024] Beneficial effects of the present invention: The present invention first processes high-resolution medical images so that they can be trained and predicted in traditional standard image recognition network models or improved models; secondly, on the basis of ensuring the high classification accuracy of the AI decision model, the present invention improves the interpretability of the image recognition network model, so that the predicted interpretable heat map can reflect the classification and recognition effect and interpretability of the high-resolution medical image; at the same time, compared with other technologies for improving interpretability, the present invention introduces professional knowledge corresponding to the interpretation area of the interpretable heat map during the training process, thereby optimizing the interpretability of the model. Finally, on the basis of improving the interpretability of the model, a method based on sparse representation is used to optimize the expression of the model's interpretability. Based on these optimization effects, the present invention can improve the credibility of medical images in model classification, and can more accurately judge the model's classification basis for medical images and the location of cancer in the medical image. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of an interpretable cancer pathology section image recognition method according to an embodiment of the present invention;

[0026] Figure 2 A modified block diagram of an image recognition network according to an embodiment of the present invention;

[0027] Figure 3 This is a training block diagram of the image recognition model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention. Figure 1 This is a flowchart of an interpretable cancer pathology slide image recognition method, such as Figure 1 As shown, the method includes:

[0029] 101. Obtain high-resolution medical images of cancer pathology sections and pre-process them to divide them into image blocks of uniform size;

[0030] In an embodiment of the present invention, the high-resolution medical image of the cancer pathology section may be a data set in an existing medical image database, or may be a high-resolution medical image collected in a hospital or other setting.

[0031] It should be noted that the high-resolution medical image herein may be a CT image, an MRI image, or a US image. Specifically, the high-resolution medical image may be a colon image, a lung image, a stomach image, a brain image, a liver image, or a heart image, and the present invention does not impose any specific limitations thereto.

[0032] In order to optimize the recognition effect and interpretability of cancer tumor images, compared with traditional technologies, the present invention also divides the training set into a prediction training set and an explanation training set; the prediction training set and the explanation training set are used to optimize the training of the image recognition network, a part of the prediction training set is mainly used for network training, the purpose of which is to make the model prediction correct, and the other part of the explanation training set is mainly used to train the network attention direction, the purpose of which is to make the model interpretation correct; in addition, the present invention also refers to the medical image to be tested used in the recognition process as a test set. For the sake of convenience of description, the verification set can be ignored in this embodiment. Those skilled in the art can choose whether to set a verification set according to actual needs, which should not be understood as a limitation of the present invention.

[0033] The embodiment of the present invention introduces high-resolution medical images in existing medical image databases as an example, but this should not be understood as a limitation of the present invention; assuming that a medical image dataset is downloaded from the TCGA and CPTAC databases, a total of 5 types of cancer tumor slices are obtained, and the patient information corresponding to each cancer tumor can be obtained at the same time, including gender and primary metastasis information. Specifically, a total of 6,324 cancer slices, including 1,328 colon cancer slices, 1,272 ovarian cancer slices, 1,647 endometrial cancer slices, 867 esophageal cancer slices, and 1,210 head and neck cancer slices, are obtained from the TCGA and CPTAC databases. The gender information in the patient information corresponding to each cancer tumor is merged with the feature vector as supplementary information during training.

[0034] In an embodiment of the present invention, another medical image preprocessing method is introduced. Whether used for medical image recognition or training, medical images that do not meet the recognition or training scenarios are first eliminated. Specifically, high-resolution medical images of cancer pathology sections are first acquired. Next, each high-resolution image of a cancer pathology section is individually determined to determine whether it meets the training or recognition scenarios. Finally, those that do not meet these scenarios are eliminated.

[0035] Of course, in practical applications, if non-CT images are mixed in with a collection of high-resolution medical images of cancer pathology slides, these high-resolution medical images of cancer pathology slides will also need to be removed. Alternatively, if the high-resolution medical images of cancer pathology slides themselves are damaged due to storage issues, these high-resolution medical images of cancer pathology slides will also need to be removed. During the verification process, image classification models can be trained to distinguish brain regions from other regions, or the statistical distribution of image pixels can be used to determine whether the high-resolution medical image is of the brain region.

[0036] Furthermore, due to factors such as shooting angle and position, most raw medical images are noisy. This noise significantly impacts medical image analysis and processing, increasing the difficulty of identifying and analyzing image details. Therefore, in embodiments of the present invention, appropriate image denoising techniques can be selected to eliminate or reduce image noise, thereby minimizing its adverse effects on subsequent medical image processing.

[0037] Because high-resolution medical images cannot be trained using standard models, the present invention also uses an image segmentation algorithm to cut cancer tumor cells from the background image and segment the cancer tumor into multiple image patches of the required size. A complete high-resolution medical image contains multiple patches, and the segmentation results are saved to a file, the file content of which is the coordinates of the segmentation result in the high-resolution medical image.

[0038] 102. Input the image block into a pre-trained feature extraction network to extract a feature vector of each image block;

[0039] In an embodiment of the present invention, the fully connected layer in the standard model is removed and only the convolutional layer is retained. Multiple image blocks of each high-resolution medical image are respectively passed through the pre-trained feature extraction network to obtain (N, D)-dimensional vectors, where n is the number of patches. The significance of this is that since high-resolution medical images cannot be trained by the standard model, the feature vectors are extracted by using the segmented patches as their pixels. Finally, the training results are saved to a file, and the file content includes the coordinates of the patches in the high-resolution image and their corresponding feature vectors.

[0040] It is understandable that in the embodiments of the present invention, the standard model refers to a traditional image recognition network, such as Resnet residual network, CNN convolutional neural network, Transformer and other standard models, as well as other improved standard models such as Densenet, Googlenet, etc.

[0041] Among them, since the high-resolution medical images are divided into a training set and a test set in the embodiment of the present invention, and the training set includes a prediction training set and an interpretation training set, the image blocks here are also correspondingly divided into prediction image blocks, interpretation image blocks and test image blocks. For the convenience of description, the first image block of the present invention corresponds to the prediction image block, the second image block corresponds to the interpretation image block, and the third image block corresponds to the test image block.

[0042] 103. Inputting the feature vector of the first image block into a preset image recognition network to pre-train the preset image recognition network;

[0043] In an embodiment of the present invention, the feature vector of the first image block is used to pre-train a preset image recognition network, wherein the preset image recognition network is an existing image recognition network, so that the image recognition network can identify the classification results of the high-resolution medical image as correctly as possible.

[0044] 104. Input the feature vector of the second image block into the pre-trained image recognition network, and output a first interpretable heat map;

[0045] In an embodiment of the present invention, the feature vector of the second image block is used to continue training the pre-trained image recognition network. Different from the traditional interpretable heat map that is interpreted according to the attention scores of the image pixels, the interpretable heat map of the high-resolution medical image is mainly interpreted according to the attention scores of the model for the image blocks, that is, the high-resolution interpretable heat map is interpreted by the attention scores of the second image blocks.

[0046] 105. Optimize the first interpretable heat map using a sparse representation method, and judge the optimized first interpretable heat map;

[0047] In this embodiment of the present invention, after generating an interpretable heatmap, a sparse representation-based method is used to further sparse the interpretation area in the original first interpretable heatmap, thereby optimizing the interpretation result. The first loss function of the sparse learning is mainly used to improve the score of the area with high attention and reduce the score of the area with low attention. The first loss function is expressed as:

[0048]

[0049] Where L(X) represents the first loss function, X represents the score of all image blocks in the current cancer pathology slice, N represents the number of image blocks in the current cancer pathology slice, and x n represents the score of the nth image block, It represents the mean score of all image blocks in the current cancer pathology slice, It represents the initial score mean of the first interpretable heat map as a whole, where the scores of all image blocks are normalized, and the loss function is optimized using the loss function minimization method, and the optimal loss result is obtained through iterative training.

[0050] In an embodiment of the present invention, the present invention also optimizes the recognition effect and the interpretability effect based on the idea of human-computer interactive learning, that is, by training the image recognition network, the network model will provide three training results including: prediction error, interpretation error; correct prediction, correct interpretation; correct prediction, interpretation error. The present invention mainly modifies the third training result in the model. When the other two types of results appear, the network is still trained. When the third training result appears, it is handed over to medical experts to judge the location of the network interpretation error, and according to the judgment of the medical experts, the image block at the corresponding position is changed. There are two results of the interpretation error judged by the medical experts. One is that the attention score of the image block location is high, indicating that the model attention is based on the wrong location prediction; the other is that the attention score of the image block location is low, indicating that the model's judgment ignores the image block at this location. Therefore, it is necessary to modify the image block according to the two results. Finally, the optimization of the first interpretable heat map is achieved.

[0051] Figure 2 is a modified block diagram of an image recognition network in an embodiment of the present invention, such as Figure 2 As shown, in an embodiment of the present invention, the training set images are input into the AI prediction model, i.e., the image recognition network, and the classification result is judged to be correct by the real label value of the training set image itself. If the classification result is inaccurate, it is necessary to return to retrain the AI prediction model until the predicted classification result reaches a certain accuracy rate. Then, the interpretable heat map corresponding to the classification result can be judged. If the interpretation area is incorrect, the AI prediction model needs to be re-corrected. Otherwise, the interpretation result can be directly output. The correction process of the image recognition network will be further explained in combination with step 106A and step 106B below:

[0052] 106A, replacing the erroneous portion of the first interpretable heat map determined to be erroneous with a counterexample, and retraining the image recognition network;

[0053] In an embodiment of the present invention, the above-mentioned judgment can identify the error area of the first interpretable graph and use the error area as a counterexample in the training sample. That is, the feature vector of the second image block is set to 0 or a counterexample, and then the cancer tumor data to which the second image block belongs is combined with the first image block to retrain the image recognition network and modify the interpretable direction of the image recognition network.

[0054] For example, since the feature vector of the second image block includes the feature vector of the wrong interpretation, it is necessary to use a counterexample for the feature vector of the wrong interpretation, that is, the feature vector of the kth wrong interpretation of the second image block Take the 0 vector Then the 0 vector The feature vector of the second image block is trained together with other data.

[0055] 106B. Alternatively, masking the erroneous portion of the first interpretable heat map that was incorrectly determined, correcting the erroneous interpretable heat map using a second loss function, and retraining the image recognition network;

[0056] In this embodiment of the present invention, the above judgment can identify the error area of the first interpretable graph, set a mask on the image block corresponding to the error portion of the first interpretable heat map, and then feed the cancer tumor data to the image block into the image recognition network for training. The image recognition network is optimized using the second loss function, thereby modifying the interpretability direction of the image recognition network. For the use of masks and the newly designed second loss function, the patch-level attention is optimized using masks, and its expression is:

[0057]

[0058] Among them, L(θ,F,y,A) represents the second loss function, θ is the network parameter, F is the input feature vector, y is the true label value, A is the mask matrix, A∈{0,1} N×D , N represents the number of image blocks in the current cancer pathology slice, K represents the number of label categories, D represents the dimension of the input feature vector, y nk represents the true value of the kth label of the nth image block, represents the k-th label prediction value of the n-th image block, A nd represents the d-th dimension mask of the n-th image block, f nd represents the d-th dimension input feature vector of the n-th image block, λ1 and λ2 are regularization factors; θ i Represents the network parameters of the i-th layer.

[0059] For example, since the feature vector of the second image block includes the feature vector of the wrong interpretation, it is necessary to use a mask and a newly designed second loss function for all feature vectors, where for the feature vectors of the S wrongly interpreted second image blocks Its mask A expFalse ∈{1} s×D , for the T correct interpretations of the feature vectors of the second image block Its mask A expTrue ∈{0}T×D .

[0060] 107. Input the feature vector of the third image block into the retrained image recognition network and output a second interpretable heat map.

[0061] In an embodiment of the present invention, the high-resolution medical image to be identified is processed to obtain a classification result and an interpretable result of the high-resolution medical image to be identified, wherein the preprocessing process and feature extraction process of the high-resolution medical image to be identified are consistent with the training process of the model. Therefore, here the feature vector of the third image block to be identified is directly used as input to obtain the classification result and interpretable result of the third image block, which are the classification result and interpretable result of the corresponding high-resolution medical image to be identified.

[0062] By training the prediction training set and the interpretation training set in result steps 103-106, an image recognition network model with correct predictions and correct interpretations can be obtained, i.e., a model with a prediction accuracy similar to that of the initial model but a higher interpretation accuracy. When a user uses the model to make predictions on high-resolution medical images of cancer pathology sections, the present invention's image recognition network based on human-computer interactive learning uses medical knowledge to achieve a two-way interaction between data and knowledge, thereby improving the interpretability of the neural network model and the user's trust in the model.

[0063] Figure 3 This is a training block diagram of the image recognition model according to an embodiment of the present invention, such as Figure 3 As shown in the figure, the entire image recognition training process includes first using the prediction training set as input, preprocessing the input prediction training set, and using the preprocessed prediction training set to train the image recognition network. At the same time, an interpretable heat map will be generated, and the interpretable heat map will be sparsely represented. The interpretable heat map after sparse representation will be given to medical experts for judgment. The interpretation training set can be updated, and the incorrectly interpreted data in the interpretation training set will be incorporated into the prediction training set for continued training, and the image recognition network will be retrained to finally obtain a network model with correct interpretation and prediction.

[0064] In an embodiment of the present invention, an interpretable cancer pathology slice image recognition device is further provided, the device comprising:

[0065] An image acquisition module for acquiring high-resolution medical images of cancer pathology sections;

[0066] An image processing module, configured to pre-process the acquired high-resolution medical image of the cancer pathology section and divide the image into image blocks of uniform size;

[0067] A feature extraction module, configured to input the image blocks into a pre-trained feature extraction network to extract a feature vector for each image block;

[0068] a pre-training module, configured to input the feature vector of the first image block into a preset image recognition network to pre-train the preset image recognition network;

[0069] an image optimization module, configured to input the feature vector of the second image block into a pre-trained image recognition network, output a first interpretable heat map, optimize the first interpretable heat map using a sparse representation method, and perform judgment on the optimized first interpretable heat map;

[0070] a retraining module, configured to replace the erroneous portion of the first interpretable heat map determined to be erroneous with a counterexample, and retrain the image recognition network; or, to provide a mask to the erroneous portion of the first interpretable heat map determined to be erroneous, correct the erroneous interpretable heat map using a second loss function, and retrain the image recognition network;

[0071] The image recognition module is configured to input the feature vector of the third image block into the retrained image recognition network and output a second interpretable heat map.

[0072] An embodiment of the present application also provides a computer device, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, wherein the computer program is used to execute any one of the implementation methods of an interpretable cancer pathology section image recognition method described in the aforementioned embodiments.

[0073] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any one of the image processing methods described in the aforementioned embodiments.

[0074] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0076] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0077] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0078] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), a random access memory (full name in English: Random Access Memory, English abbreviation: RAM), a magnetic disk or an optical disk, etc. Various media that can store program code. Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, a magnetic disk or an optical disk, etc.

[0079] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for interpretable cancer pathology slide image recognition, characterized in that: The method comprises: Obtain high-resolution medical images of cancer pathology sections and preprocess them to divide them into image blocks of uniform size; Inputting the image block into a pre-trained feature extraction network to extract a feature vector of each image block; Inputting the feature vector of the first image block into a preset image recognition network to pre-train the preset image recognition network; Inputting the feature vector of the second image block into the pre-trained image recognition network and outputting a first interpretable heat map; Performing a sparse representation on the interpretation region of the first interpretable heat map, calculating the attention score of the second image block, increasing the score of the interpretation region corresponding to high attention, and decreasing the score of the interpretation region corresponding to low attention; optimizing the interpretation region of the first interpretable heat map, and judging the optimized first interpretable heat map; replacing the erroneous portion of the first interpretable heat map determined to be erroneous with a counterexample, and retraining the image recognition network; Alternatively, providing a mask for an erroneous portion of the first interpretable heat map that was incorrectly determined, correcting the erroneous interpretable heat map using a second loss function, and retraining the image recognition network; The feature vector of the third image patch is input into the retrained image recognition network, and a second interpretable heat map is output.

2. The interpretable cancer pathology slice image recognition method according to claim 1, characterized in that: Preprocessing the high-resolution medical image of the cancer pathology section includes segmenting the cancer tumor into multiple image blocks of a desired size according to an image segmentation algorithm, and saving the segmentation results to a file, the file content of which is the coordinates of the segmentation results in the high-resolution medical image.

3. The interpretable cancer pathology slice image recognition method according to claim 1, characterized in that: The first loss function used by the sparse representation method to optimize the first interpretable heat map is expressed as: Where L(X) represents the first loss function, X represents the score of all image blocks in the current cancer pathology slice, N represents the number of image blocks in the current cancer pathology slice, and x n represents the score of the nth image block, It represents the mean score of all image blocks in the current cancer pathology slice, Represents the initial score mean of the first interpretable heatmap as a whole.

4. The interpretable cancer pathology slice image recognition method according to claim 1, characterized in that: The erroneous portion of the first interpretable heat map that is judged to be erroneous is replaced with a counterexample, and the image recognition network is retrained, including setting the feature vector of the image block corresponding to the erroneous portion of the first interpretable heat map to 0 or taking a counterexample, and then sending the cancer tumor data to the image block to be trained into the image recognition network, thereby modifying the interpretable direction of the image recognition network.

5. The interpretable cancer pathology slice image recognition method according to claim 1, characterized in that: A mask is provided for the erroneous portion of the first interpretable heat map that is judged to be erroneous, and the erroneous interpretable heat map is corrected using a second loss function. The image recognition network is retrained, including setting a mask for the image block corresponding to the erroneous portion of the first interpretable heat map, and then sending the cancer tumor data to the image block into the image recognition network for training, and optimizing the image recognition network using the second loss function, thereby modifying the interpretable direction of the image recognition network.

6. The interpretable cancer pathology slice image recognition method according to claim 1 or 5, characterized in that: The second loss function is expressed as: Among them, L(θ,F,y,A) represents the second loss function, θ is the network parameter, F is the input feature vector, y is the true label value, A is the mask matrix, N represents the number of image blocks in the current cancer pathology slice, K represents the number of label categories, D represents the dimension of the input feature vector, and y nk represents the true value of the kth label of the nth image block, A represents the k-th label prediction value of the n-th image block, nd represents the d-th dimension mask of the n-th image block, f nd represents the d-th dimension input feature vector of the n-th image block, λ1 and λ2 are regularization factors; θ i Represents the network parameters of the i-th layer.

7. An interpretable cancer pathology slice image recognition device, used to implement an interpretable cancer pathology slice image recognition method according to any one of claims 1 to 6, characterized in that: The device comprises: An image acquisition module for acquiring high-resolution medical images of cancer pathology sections; An image processing module, configured to pre-process the acquired high-resolution medical image of the cancer pathology section and divide the image into image blocks of uniform size; A feature extraction module, configured to input the image blocks into a pre-trained feature extraction network to extract a feature vector for each image block; a pre-training module, configured to input the feature vector of the first image block into a preset image recognition network to pre-train the preset image recognition network; an image optimization module, configured to input the feature vector of the second image block into a pre-trained image recognition network, output a first interpretable heat map, optimize the first interpretable heat map using a sparse representation method, and perform judgment on the optimized first interpretable heat map; a retraining module, configured to replace the erroneous portion of the first interpretable heat map determined to be erroneous with a counterexample, and retrain the image recognition network; or, to provide a mask to the erroneous portion of the first interpretable heat map determined to be erroneous, correct the erroneous interpretable heat map using a second loss function, and retrain the image recognition network; The image recognition module is configured to input the feature vector of the third image block into the retrained image recognition network and output a second interpretable heat map.

8. The interpretable cancer pathology slice image recognition device according to claim 7, characterized in that: The device further comprises: The interactive learning module is used to judge the optimized first interpretable heat map to determine whether the recognition prediction result corresponding to the first interpretable heat map is correct and whether the interpretation area is correct.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the identification method according to any one of claims 1 to 6.

Images