An intelligent auxiliary diagnosis method for breast tissue pathology based on multi-label learning
The method enhances breast tissue pathology diagnosis by combining multi-label learning with convolutional neural networks and multi-instance learning to address inefficiencies in existing technologies, improving the accuracy of multi-class classification and multi-class classification of breast cancer types.
Patent Information
- Application Number
- CN202410906969.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2044-07-08
AI Technical Summary
Current multi-label classification methods for breast tissue pathology images are inefficient and inaccurate due to reliance on weakly supervised frameworks, inconsistent feature extraction, and region-of-interest based learning, which fail to adapt to different data types and tasks, and cannot effectively handle the complex and varied nature of breast cancer types.
A multi-label learning-based method involving binary classification and large/small annotation of breast tissue pathology images, combined with convolutional neural networks and multi-instance learning, to extract and integrate discriminative features for accurate diagnosis.
Improves the accuracy of multi-label classification and auxiliary diagnosis of breast diseases by effectively integrating large and small annotation features, enhancing the model's ability to learn and adapt to diverse breast cancer types.
Smart Images
Figure CN118887170B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to an intelligent auxiliary diagnosis method for breast tissue pathology based on multi-label learning. Background Art
[0002] Breast cancer is the most common malignancy in women worldwide. Pathologists extract quantitative data from histological samples and determine the grade, type, distribution and tumor stage of tumor cells, or biomarkers such as hormone receptors, gene evaluation and proliferation index. These biological characteristics provide important information about the type of breast cancer, thereby providing the most appropriate treatment for each case. The complex information and widely varying contents in histopathology images make this process a highly specialized task that requires experienced pathologists to complete, but its efficiency is not higher than that of traditional microscopic examination. The whole process is time-consuming, costly, and susceptible to factors such as decreased attention and fatigue, which has a significant impact on the treatment effect of patients. In view of this, there is an urgent need to design intelligent auxiliary diagnosis methods for breast tissue pathology full-slice images, so as to perform efficient automatic analysis of large breast tissue pathology images.
[0003] Breast cancer is a complex tumor, with specific types of diseases covering more than 100 categories and showing a long-tail distribution. In the treatment of breast cancer, professional pathologists usually quickly and accurately judge the patient's condition and formulate corresponding treatment plans based on multi-classification results. Therefore, the auxiliary diagnosis method for breast tissue pathology full-slice images needs to complete multi-label classification tasks. In the existing weakly supervised image classification algorithms, the method adopted is to design a classifier to perform binary classification of breast tissue pathology images by using pathology full slices with regions of interest, that is, the results of the judgment are benign or malignant. This method cannot meet the diagnostic needs of breast diseases with multiple disease categories.
[0004] The main solution to this problem is to effectively combine multi-instance learning and multi-label learning. In the training process of multi-instance learning, no label is assigned to each instance, while the training samples of multi-label learning can be associated with multiple labels to describe the samples in multiple forms. After effectively combining the above two learning methods, each training sample is represented by a package of multiple instances, and each package is assigned multiple categories of labels.
[0005] Most of the current multi-label classification methods for breast tissue pathology images are based on the classification of pathology full-slice images under a weakly supervised framework. This method cannot fully learn image features that are beneficial to the classification of pathology full-slices. Moreover, most of them are based on feature extraction using trained feature extractors, which results in the model's performance being unable to adapt to its own classification tasks. It has the following two disadvantages:
[0006] First, the features extracted by the above method are calculated by color histogram, texture histogram and cell nucleus detection results, rather than the features obtained by the deep learning model. This feature extraction method cannot adapt to different data types and tasks.
[0007] Second, during its learning process, each instance corresponds to the region of interest of the breast tissue pathology image, rather than the image at the small slice level. It is derived from the important potential information region extracted by the pathologist when viewing the full slice image of breast tissue pathology. However, the size of each region of interest image is different, and the image size often needs to be reset during the learning process, which will lead to inconsistent receptive fields for learning each region of interest image, affecting the performance of the model. Therefore, the basic unit corresponding to each instance cannot accurately learn the image features in the lesion area, thus affecting the results of classification diagnosis. Summary of the invention
[0008] The present invention provides a breast tissue pathology intelligent auxiliary diagnosis method based on multi-label learning to solve the problems raised in the background technology.
[0009] A multi-label learning-based intelligent auxiliary diagnosis method for breast tissue pathology, comprising:
[0010] S1: Perform binary classification large annotation on the pathological full-slice image of breast tissue to obtain a first-class label, divide the pathological full-slice image of breast tissue into a number of identical image slice blocks, perform small annotation on the image slice blocks, and obtain a second-class label;
[0011] S2: Obtain the first-class labels and second-class labels of the pathological full-slice image of breast tissue to train the feature extractor, and obtain the large-label features and small-label features of the target pathological full-slice image based on the feature training results;
[0012] S3: training the convolutional neural network model and the multiple instance learning model based on the large annotation features and the small annotation features to obtain the target convolutional neural network model and the target multiple instance learning model;
[0013] S4: The full-slice image of the pathology to be detected is input into the target convolutional neural network model and the target multi-instance learning model respectively, and intelligent auxiliary diagnosis is performed according to the output results.
[0014] Preferably, in S1, the pathological full-slice image of breast tissue is subjected to binary classification and large annotation to obtain a class of labels, including:
[0015] Acquire the mild pathology area and the severe pathology area in the pathological full-slice image of breast tissue;
[0016] The first label is assigned to the mild pathology area, and the second label is assigned to the severe pathology area;
[0017] The first tag and the second tag form a type of tag.
[0018] Preferably, in S1, the pathological full-slice image of breast tissue is divided into a plurality of identical image slice blocks, and the image slice blocks are annotated to obtain two types of labels, including:
[0019] Dividing the pathological full-slice image of breast tissue into a number of identical image slice blocks according to a preset block size;
[0020] Based on the label features of a class of labels of the pathological full-slice image, a label weight is assigned to each image slice block;
[0021] A preset number of target image slice blocks are randomly extracted from all image slice blocks according to label weights;
[0022] Based on the specific category of breast cancer, the target slice block is annotated with the specific lesion type to obtain a second category label.
[0023] Preferably, in S2, obtaining the first-class label and the second-class label of the pathological full-slice image of breast tissue to train the feature extractor includes:
[0024] Initialize the feature extractor using the parameters of the pre-trained feature extractor using a preset data set;
[0025] Based on the first-class label, the feature extractor extracts features from the pathological full-slice image to obtain a first-class feature vector, randomly selects a first-class feature vector as a first-class reference vector, obtains a first-class vector difference value between other first-class feature vectors and the first-class reference vector, and determines a first-class function loss value based on the first-class vector difference value;
[0026] Based on the feature extractor, feature extraction is performed on the pathological full slices with the same first-class label and different second-class labels to obtain a second-class feature vector, a second-class feature vector is randomly selected as a second-class reference vector, the second-class vector difference value between other second-class feature vectors and the second-class reference vector is obtained, and the first second-class function loss value is determined based on the first-class vector difference value;
[0027] Based on the feature extractor, feature extraction is performed on the pathological full slices with different first-class labels and different second-class labels to obtain three-class feature vectors, one of the three-class feature vectors is randomly selected as the three-class reference vector, the three-class vector difference values between the other three-class feature vectors and the three-class reference vectors are obtained, and the second-class function loss value is determined based on the three-class vector difference values;
[0028] The first weight is determined based on the number of pathological full sections with the same one-category label and two-category label differences, and the second weight is determined based on the number of pathological full sections with different one-category labels and two-category label differences;
[0029] Determine a second class function loss value based on a first second class function loss value and a first weight, a second second class function loss value, and a second weight;
[0030] Based on the first-class function loss value and the second-class function loss value, the feature extractor is iteratively trained using the pathological full-slice image to obtain the target feature extractor.
[0031] Preferably, in S2, based on the feature training result, large annotation features and small annotation features are obtained for the target pathological full-slice image, including:
[0032] Based on the target feature extractor, the target pathology full-slice image is encoded with a class of features to obtain a 384-dimensional class of feature vectors, and a large annotation feature is obtained based on the class of feature vectors;
[0033] Based on the target feature extractor, the target pathology full-slice image is encoded with two-category features to obtain a 384-dimensional two-category feature vector, and small annotation features are obtained based on the two-category feature vector.
[0034] Preferably, in S3, the convolutional neural network model is trained based on the large annotation features and the small annotation features to obtain a target convolutional neural network model, including:
[0035] Iteratively train the convolutional neural network model based on the large annotation features and the small annotation features to obtain the large annotation loss value and the small annotation loss value of each iterative training;
[0036] Add the large annotation loss value and the small annotation loss value and perform back propagation update to perform the next iterative training;
[0037] The model is verified for each iterative training result based on the preset validation set until the model performance no longer improves, and the target convolutional neural network model is obtained.
[0038] Preferably, in S3, training a multi-instance learning model based on large annotation features and small annotation features to obtain a target multi-instance learning model includes:
[0039] S301: inputting the large annotation features and the small annotation features into a multi-instance learning model, and mapping each annotation feature into a sequence vector through linear projection;
[0040] S302: interactively modeling the sequence vectors based on the self-attention mechanism in the multi-instance learning model to obtain an information weight matrix;
[0041] S303: Mapping and classifying the information weight matrix through the classification layer to obtain an output label matrix, and obtaining a training result based on the output label matrix;
[0042] S304: Iteratively train the large annotation features and the small annotation features according to the method of S301-S303, and perform model verification on each training result based on a preset verification set until the model performance no longer improves, thereby obtaining a target multi-instance learning model.
[0043] Preferably, in S302, the sequence vectors are interactively modeled based on the self-attention mechanism in the multi-instance learning model to obtain an information weight matrix, including:
[0044] Mapping the sequence vector through an embedding layer to obtain intermediate sequence features;
[0045] The intermediate sequence features are respectively transformed through a query transformation matrix, a key value transformation demonstration and an information change matrix to obtain a query matrix, a key value matrix and an information matrix;
[0046] Based on the query matrix, key value matrix and information matrix, the information weight matrix is obtained according to the following formula;
[0047]
[0048] Among them, N represents the information weight matrix, softmax() represents the normalized exponential function, T represents a constant used to adjust the smoothness of the output, Q represents the query matrix, K represents the key value matrix, V represents the information matrix, and d represents the vector length of the intermediate key value matrix.
[0049] Preferably, in S303, mapping and classifying the information weight matrix through a classification layer to obtain an output label matrix, and obtaining a training result based on the output label matrix includes:
[0050] Mapping the information weight matrix through the pooling layer and the normalization layer in the classification layer in turn, and outputting an output label matrix;
[0051] The training results are obtained based on the comparison between the label features in the output label matrix and the large annotation features and the small annotation features.
[0052] Preferably, in S4, the full-slice image of the pathology to be detected is input into the target convolutional neural network model and the target multiple instance learning model respectively, and intelligent auxiliary diagnosis is performed according to the output results, including:
[0053] The pathological full-slice image to be detected is input into the target convolutional neural network model and the target multiple instance learning model respectively to obtain a first image annotation classification result and a second image annotation classification result;
[0054] According to the degree of annotation, the first image annotation classification result and the second image annotation classification result are integrated to obtain the target image annotation classification result;
[0055] The target image annotation classification result is input into the intelligent auxiliary diagnosis model to obtain an auxiliary diagnosis result.
[0056] Compared with the prior art, the present invention has achieved the following beneficial effects:
[0057] The pathological full-slice image of breast tissue is subjected to binary classification and large annotation to obtain a class of labels, the pathological full-slice image of breast tissue is divided into several identical image slice blocks, the image slice blocks are subjected to small annotation to obtain a class of labels, the class of labels and the class of labels of the pathological full-slice image of breast tissue are taken to train the feature extractor, based on the feature training results, large annotation features and small annotation features are obtained for the target pathological full-slice image, the convolutional neural network model and the multi-instance learning model are trained based on the large annotation features and the small annotation features to obtain the target convolutional neural network model and the target multi-instance learning model, the pathological full-slice image to be detected is respectively input into the target convolutional neural network model and the target multi-instance learning model, and intelligent auxiliary diagnosis is performed according to the output results, by using large annotations and small annotations, the feature extractor and the multi-instance learning model are trained at the same time, and more discriminative feature information can be fully integrated, so that the model can learn better features, thereby improving the accuracy of multi-label classification of breast tissue pathological images and the accuracy of auxiliary diagnosis of breast diseases.
[0058] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.
[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0061] Figure 1 This is a flow chart of a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning in an embodiment of the present invention;
[0062] Figure 2 A flowchart of obtaining a target convolutional neural network model in an embodiment of the present invention;
[0063] Figure 3 The flowchart of obtaining the target multi-instance learning model in an embodiment of the present invention. DETAILED DESCRIPTION
[0064] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0065] Embodiment 1:
[0066] The embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning, such as Figure 1 As shown, including:
[0067] S1: Perform binary classification large annotation on the pathological full-slice image of breast tissue to obtain a first-class label, divide the pathological full-slice image of breast tissue into a number of identical image slice blocks, perform small annotation on the image slice blocks, and obtain a second-class label;
[0068] S2: Obtain the first-class labels and second-class labels of the pathological full-slice image of breast tissue to train the feature extractor, and obtain the large-label features and small-label features of the target pathological full-slice image based on the feature training results;
[0069] S3: training the convolutional neural network model and the multiple instance learning model based on the large annotation features and the small annotation features to obtain the target convolutional neural network model and the target multiple instance learning model;
[0070] S4: The full-slice image of the pathology to be detected is input into the target convolutional neural network model and the target multi-instance learning model respectively, and intelligent auxiliary diagnosis is performed according to the output results.
[0071] The beneficial effects of the above design scheme are as follows: by performing binary classification large annotation on the pathological full-slice image of breast tissue, a first-class label is obtained; the pathological full-slice image of breast tissue is divided into several identical image slice blocks, and small annotation is performed on the image slice blocks to obtain a second-class label; the first-class label and the second-class label of the pathological full-slice image of breast tissue are used to train the feature extractor; based on the feature training result, large annotation features and small annotation features are obtained for the target pathological full-slice image; based on the large annotation features and the small annotation features, a convolutional neural network model and a multi-instance learning model are trained to obtain a target convolutional neural network model and a target multi-instance learning model; the pathological full-slice image to be detected is respectively input into the target convolutional neural network model and the target multi-instance learning model; intelligent auxiliary diagnosis is performed according to the output results; by using large annotations and small annotations, the feature extractor and the multi-instance learning model are trained at the same time, so that more discriminative feature information can be fully integrated, so that the model can learn better features, thereby improving the accuracy of multi-label classification of breast tissue pathological images and improving the accuracy of auxiliary diagnosis of breast diseases.
[0072] Embodiment 2:
[0073] Based on Example 1, the embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning. In S1, the pathological full-slice image of breast tissue is labeled by two categories to obtain a class of labels, including:
[0074] Acquire the mild pathology area and the severe pathology area in the pathological full-slice image of breast tissue;
[0075] The first label is assigned to the mild pathology area, and the second label is assigned to the severe pathology area;
[0076] The first tag and the second tag form a type of tag.
[0077] The beneficial effect of the above design scheme is: by obtaining the light pathology area and the heavy pathology area in the pathology full slice image of breast tissue, the light pathology area is assigned a first label, and the heavy pathology area is assigned a second label; the first label and the second label form a class of labels to achieve large-scale labeling of the pathology full slice image of breast tissue.
[0078] Embodiment 3:
[0079] Based on Example 2, the embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning. In S1, the pathological full slice image of breast tissue is divided into a plurality of identical image slice blocks, and the image slice blocks are annotated to obtain two types of labels, including:
[0080] Dividing the pathological full-slice image of breast tissue into a number of identical image slice blocks according to a preset block size;
[0081] Based on the label features of a class of labels of the pathological full-slice image, a label weight is assigned to each image slice block;
[0082] A preset number of target image slice blocks are randomly extracted from all image slice blocks according to label weights;
[0083] Based on the specific category of breast cancer, the target slice block is annotated with the specific lesion type to obtain a second category label.
[0084] The beneficial effect of the above design scheme is: by randomly extracting a preset number of target image slice blocks from all image slice blocks according to the label weight, based on the specific category of breast cancer, small annotations of specific lesion types are made to the target slice blocks to obtain second-category labels, and the labeling efficiency is guaranteed based on a small amount of annotations, while providing small-level annotation features, which provides a basis for improving the accuracy of image-assisted diagnosis of breast tissue pathology.
[0085] Embodiment 4:
[0086] Based on Example 1, the embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning. In S2, the first and second labels of the pathological full-slice image of the breast tissue are obtained to train the feature extractor, including:
[0087] Initialize the feature extractor using the parameters of the pre-trained feature extractor using a preset data set;
[0088] Based on the first-class label, the feature extractor extracts features from the pathological full-slice image to obtain a first-class feature vector, randomly selects a first-class feature vector as a first-class reference vector, obtains a first-class vector difference value between other first-class feature vectors and the first-class reference vector, and determines a first-class function loss value based on the first-class vector difference value;
[0089] Based on the feature extractor, feature extraction is performed on the pathological full slices with the same first-class label and different second-class labels to obtain a second-class feature vector, a second-class feature vector is randomly selected as a second-class reference vector, the second-class vector difference value between other second-class feature vectors and the second-class reference vector is obtained, and the first second-class function loss value is determined based on the first-class vector difference value;
[0090] Based on the feature extractor, feature extraction is performed on the pathological full slices with different first-class labels and different second-class labels to obtain three-class feature vectors, one of the three-class feature vectors is randomly selected as the three-class reference vector, the three-class vector difference values between the other three-class feature vectors and the three-class reference vectors are obtained, and the second-class function loss value is determined based on the three-class vector difference values;
[0091] The first weight is determined based on the number of pathological full sections with the same one-category label and two-category label differences, and the second weight is determined based on the number of pathological full sections with different one-category labels and two-category label differences;
[0092] Determine a second class function loss value based on a first second class function loss value and a first weight, a second second class function loss value, and a second weight;
[0093] Based on the first-class function loss value and the second-class function loss value, the feature extractor is iteratively trained using the pathological full-slice image to obtain the target feature extractor.
[0094] The beneficial effect of the above design scheme is: by determining the loss value of different situations based on the type of label, and analyzing and processing based on the loss value, the target feature extractor is finally obtained, and the accuracy of the target feature extractor is guaranteed, so that it can extract more conducive to extracting classification image features, providing a basis for intelligent auxiliary diagnosis of breast tissue pathology.
[0095] Embodiment 5:
[0096] Based on Example 4, the embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning. In S2, based on the feature training result, large annotation features and small annotation features are obtained for the target pathology full slice image, including:
[0097] Based on the target feature extractor, the target pathology full-slice image is encoded with a class of features to obtain a 384-dimensional class of feature vectors, and a large annotation feature is obtained based on the class of feature vectors;
[0098] Based on the target feature extractor, the target pathology full-slice image is encoded with two-category features to obtain a 384-dimensional two-category feature vector, and small annotation features are obtained based on the two-category feature vector.
[0099] The beneficial effects of the above design scheme are: based on the target feature extractor, a first-class feature encoding is performed on the target pathology full-slice image to obtain a 384-dimensional first-class feature vector, and a large annotation feature is obtained based on the first-class feature vector. Based on the target feature extractor, a second-class feature encoding is performed on the target pathology full-slice image to obtain a 384-dimensional second-class feature vector, and a small annotation feature is obtained based on the second-class feature vector, thereby ensuring the comprehensiveness and accuracy of the obtained large annotation features and small annotation features, and providing a basis for intelligent auxiliary diagnosis of breast tissue pathology.
[0100] Embodiment 6:
[0101] Based on Example 1, the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning, such as Figure 2As shown, in S3, the convolutional neural network model is trained based on the large annotation features and the small annotation features to obtain the target convolutional neural network model, including:
[0102] Iteratively train the convolutional neural network model based on the large annotation features and the small annotation features to obtain the large annotation loss value and the small annotation loss value of each iterative training;
[0103] Add the large annotation loss value and the small annotation loss value and perform back propagation update to perform the next iterative training;
[0104] The model is verified for each iterative training result based on the preset validation set until the model performance no longer improves, and the target convolutional neural network model is obtained.
[0105] The beneficial effect of the above design scheme is: by training the convolutional neural network model based on large annotation features and small annotation features, a target convolutional neural network model is obtained, which provides a neural network model basis for breast pathology diagnosis.
[0106] Embodiment 7:
[0107] Based on Example 1, the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning, such as Figure 3 As shown, in S3, the multi-instance learning model is trained based on the large annotation features and the small annotation features to obtain the target multi-instance learning model, including:
[0108] S301: inputting the large annotation features and the small annotation features into a multi-instance learning model, and mapping each annotation feature into a sequence vector through linear projection;
[0109] S302: interactively modeling the sequence vectors based on the self-attention mechanism in the multi-instance learning model to obtain an information weight matrix;
[0110] S303: Mapping and classifying the information weight matrix through the classification layer to obtain an output label matrix, and obtaining a training result based on the output label matrix;
[0111] S304: Iteratively train the large annotation features and the small annotation features according to the method of S301-S303, and perform model verification on each training result based on a preset verification set until the model performance no longer improves, thereby obtaining a target multi-instance learning model.
[0112] The beneficial effect of the above design scheme is: by training the multi-instance learning model based on large annotation features and small annotation features, a target multi-instance learning model is obtained, which provides a multi-instance learning model basis for breast pathology diagnosis.
[0113] Embodiment 8:
[0114] Based on Example 7, the embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning. In S302, the sequence vector is interactively modeled based on the self-attention mechanism in the multi-instance learning model to obtain an information weight matrix, including:
[0115] Mapping the sequence vector through an embedding layer to obtain intermediate sequence features;
[0116] The intermediate sequence features are respectively transformed through a query transformation matrix, a key value transformation demonstration and an information change matrix to obtain a query matrix, a key value matrix and an information matrix;
[0117] Based on the query matrix, key value matrix and information matrix, the information weight matrix is obtained according to the following formula;
[0118]
[0119] Among them, N represents the information weight matrix, softmax() represents the normalized exponential function, T represents a constant used to adjust the smoothness of the output, Q represents the query matrix, K represents the key value matrix, V represents the information matrix, and d represents the vector length of the intermediate key value matrix.
[0120] The beneficial effect of the above design scheme is: the multi-instance learning model is trained and learned through the self-attention mechanism, which ensures the comprehensiveness and accuracy of the information weight matrix, thereby ensuring the performance of the target multi-instance learning model.
[0121] Embodiment 9:
[0122] Based on Example 7, the embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning. In S303, the information weight matrix is mapped and classified by the classification layer to obtain an output label matrix, and the training result is obtained based on the output label matrix, including:
[0123] Mapping the information weight matrix through the pooling layer and the normalization layer in the classification layer in turn, and outputting an output label matrix;
[0124] The training results are obtained based on the comparison between the label features in the output label matrix and the large annotation features and the small annotation features.
[0125] The beneficial effect of the above design scheme is: the information weight matrix is mapped and classified through the classification layer to obtain an output label matrix, and the training results are obtained based on the output label matrix to realize the training of the multi-instance learning model.
[0126] Embodiment 10:
[0127] Based on Example 1, the embodiment of the present invention provides a method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning. In S4, the full-slice image of the pathology to be detected is input into the target convolutional neural network model and the target multi-instance learning model respectively, and intelligent auxiliary diagnosis is performed according to the output results, including:
[0128] The pathological full-slice image to be detected is input into the target convolutional neural network model and the target multiple instance learning model respectively to obtain a first image annotation classification result and a second image annotation classification result;
[0129] According to the degree of annotation, the first image annotation classification result and the second image annotation classification result are integrated to obtain the target image annotation classification result;
[0130] The target image annotation classification result is input into the intelligent auxiliary diagnosis model to obtain an auxiliary diagnosis result.
[0131] The beneficial effect of the above design scheme is that through the effective combination of the target convolutional neural network model and the target multi-instance learning model, it is possible to fully integrate more discriminative feature information, so that the model can learn better features, thereby improving the accuracy of multi-label classification of breast tissue pathology images and improving the accuracy of auxiliary diagnosis of breast diseases.
[0132] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of this application document and its equivalent technology, the present invention is also intended to include these changes and variations.
Claims
1. A method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning, characterized in that: include: S1: Perform binary classification large annotation on the pathological full-slice image of breast tissue to obtain a first-class label, divide the pathological full-slice image of breast tissue into a number of identical image slice blocks, perform small annotation on the image slice blocks, and obtain a second-class label; S2: Obtain the first-class labels and second-class labels of the pathological full-slice image of breast tissue to train the feature extractor, and obtain the large-label features and small-label features for the target pathological full-slice image based on the feature training results; S3: training the convolutional neural network model and the multiple instance learning model based on the large annotation features and the small annotation features to obtain the target convolutional neural network model and the target multiple instance learning model; S4: Input the full-slice image of the pathology to be detected into the target convolutional neural network model and the target multi-instance learning model respectively, and perform intelligent auxiliary diagnosis according to the output results; In S1, the pathological full-slice image of breast tissue is annotated by binary classification to obtain a class of labels, including: Acquire the mild pathology area and the severe pathology area in the pathological full-slice image of breast tissue; The first label is assigned to the mild pathology area, and the second label is assigned to the severe pathology area; The first label and the second label form a type of label; In S1, the pathological full-slice image of breast tissue is divided into a number of identical image slice blocks, and the image slice blocks are annotated to obtain two types of labels, including: Dividing the pathological full-slice image of breast tissue into a number of identical image slice blocks according to a preset block size; Based on the label features of a class of labels of the pathological full-slice image, a label weight is assigned to each image slice block; A preset number of target image slice blocks are randomly extracted from all image slice blocks according to label weights; Based on the specific category of breast cancer, the target slice block is annotated with the specific lesion type to obtain a second-category label; In S2, the first-class label and the second-class label of the pathological full-slice image of breast tissue are obtained to train the feature extractor, including: Initialize the feature extractor using the parameters of the pre-trained feature extractor using a preset data set; Based on the first-class label, the feature extractor extracts features from the pathological full-slice image to obtain a first-class feature vector, randomly selects a first-class feature vector as a first-class reference vector, obtains a first-class vector difference value between other first-class feature vectors and the first-class reference vector, and determines a first-class function loss value based on the first-class vector difference value; Based on the feature extractor, feature extraction is performed on the pathological full slices with the same first-class label and different second-class labels to obtain a second-class feature vector, a second-class feature vector is randomly selected as a second-class reference vector, the second-class vector difference value between other second-class feature vectors and the second-class reference vector is obtained, and the first second-class function loss value is determined based on the second-class vector difference value; Based on the feature extractor, feature extraction is performed on the pathological full slices with different first-class labels and different second-class labels to obtain three-class feature vectors, one of the three-class feature vectors is randomly selected as the three-class reference vector, the three-class vector difference values between the other three-class feature vectors and the three-class reference vectors are obtained, and the second-class function loss value is determined based on the three-class vector difference values; The first weight is determined based on the number of pathological full sections with the same one-category label and two-category label differences, and the second weight is determined based on the number of pathological full sections with different one-category labels and two-category label differences; Determine a second class function loss value based on a first second class function loss value and a first weight, a second second class function loss value, and a second weight; Based on the first-class function loss value and the second-class function loss value, the feature extractor is iteratively trained using the pathological full-slice image to obtain the target feature extractor.
2. According to claim 1, a breast tissue pathology intelligent auxiliary diagnosis method based on multi-label learning is characterized in that: In S2, based on the feature training results, large annotation features and small annotation features are obtained for the target pathological full-slice image, including: Based on the target feature extractor, the target pathology full-slice image is encoded with a class of features to obtain a 384-dimensional class of feature vectors, and a large annotation feature is obtained based on the class of feature vectors; Based on the target feature extractor, the target pathology full-slice image is encoded with two-category features to obtain a 384-dimensional two-category feature vector, and small annotation features are obtained based on the two-category feature vector.
3. According to claim 1, a breast tissue pathology intelligent auxiliary diagnosis method based on multi-label learning is characterized in that: In S3, the convolutional neural network model is trained based on the large annotation features and the small annotation features to obtain a target convolutional neural network model, including: Iteratively train the convolutional neural network model based on the large annotation features and the small annotation features to obtain the large annotation loss value and the small annotation loss value of each iterative training; Add the large annotation loss value and the small annotation loss value and perform back propagation update to perform the next iterative training; The model is verified for each iterative training result based on the preset validation set until the model performance no longer improves, and the target convolutional neural network model is obtained.
4. The method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning according to claim 1, characterized in that: In S3, the multi-instance learning model is trained based on the large annotation features and the small annotation features to obtain a target multi-instance learning model, including: S301: inputting the large annotation features and the small annotation features into a multi-instance learning model, and mapping each annotation feature into a sequence vector through linear projection; S302: interactively modeling the sequence vectors based on the self-attention mechanism in the multi-instance learning model to obtain an information weight matrix; S303: Mapping and classifying the information weight matrix through the classification layer to obtain an output label matrix, and obtaining a training result based on the output label matrix; S304: Iteratively train the large annotation features and the small annotation features according to the method of S301-S303, and perform model verification on each training result based on a preset verification set until the model performance no longer improves, thereby obtaining a target multi-instance learning model.
5. The method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning according to claim 4 is characterized in that: In S302, the sequence vectors are interactively modeled based on the self-attention mechanism in the multi-instance learning model to obtain an information weight matrix, including: Mapping the sequence vector through an embedding layer to obtain intermediate sequence features; The intermediate sequence features are transformed by a query transformation matrix, a key value transformation matrix and an information change matrix to obtain a query matrix, a key value matrix and an information matrix; Based on the query matrix, key value matrix and information matrix, the information weight matrix is obtained according to the following formula; in, represents the information weight matrix, () represents the normalized exponential function, Represents a constant, used to adjust the smoothness of the output. represents the query matrix, K represents the key-value matrix, V represents the information matrix, and d represents the vector length of the intermediate key-value matrix.
6. The method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning according to claim 4, characterized in that: In S303, mapping and classifying the information weight matrix through the classification layer to obtain an output label matrix, and obtaining a training result based on the output label matrix includes: Mapping the information weight matrix through the pooling layer and the normalization layer in the classification layer in turn, and outputting an output label matrix; The training results are obtained based on the comparison between the label features in the output label matrix and the large annotation features and the small annotation features.
7. The method for intelligent auxiliary diagnosis of breast tissue pathology based on multi-label learning according to claim 1, characterized in that: In S4, the full-slice image of the pathology to be detected is input into the target convolutional neural network model and the target multi-instance learning model respectively, and intelligent auxiliary diagnosis is performed according to the output results, including: The pathological full-slice image to be detected is input into the target convolutional neural network model and the target multiple instance learning model respectively to obtain a first image annotation classification result and a second image annotation classification result; According to the degree of annotation, the first image annotation classification result and the second image annotation classification result are integrated to obtain the target image annotation classification result; The target image annotation classification result is input into the intelligent auxiliary diagnosis model to obtain an auxiliary diagnosis result.