Partial multi-label medical image identification method based on pseudo-label identification strategy

Abstract

The invention discloses a partial multi-label medical image recognition method based on a pseudo-label recognition strategy, and relates to the technical field of image processing. The method comprises the steps that an image recognition model and a training data set are constructed, and training based on an asymmetric double-threshold pseudo-label recognition strategy and a label condition co-occurrence relation pseudo-label recognition strategy is carried out on the image recognition model; and obtaining a to-be-recognized medical image, carrying out preprocessing, inputting the preprocessed medical image into the trained image recognition model, and carrying out feature extraction and classification recognition to obtain an image recognition result. According to the method, high-quality pseudo tags are preliminarily generated by using an asymmetric dual-threshold pseudo tag identification strategy, potential positive tags in unknown tags are further identified by capturing a co-occurrence relationship among the tags, and the problem of imbalance between the positive and negative tags is solved by training a model by using PASL Loss. Therefore, the image recognition precision is higher under the condition of label missing, and the accuracy and reliability of medical diagnosis are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a partial multi-label medical image recognition method based on a pseudo-label recognition strategy. Background Art

[0002] Traditional multi-label medical image recognition methods usually rely on large-scale and fully annotated data sets. However, the high annotation cost and the complexity of medical images make it extremely difficult to collect a complete annotated data set. Therefore, in real clinical scenarios, most medical image data sets only have partial annotations and it is difficult to comprehensively cover all possible label information. Therefore, deep learning medical image recognition methods based on partial multi-labels have received extensive attention in recent years.

[0003] To address the problem of multi-label image recognition under missing labels, many solutions have been proposed in the academic and industrial communities at home and abroad. Among the technical solutions relatively close to the present invention are: T. Chen et al. (Chen T, Pu T, Liu L, et al. Heterogeneous semantic transfer for multi-label recognition with partial labels[J]. International Journal of Computer Vision, 2024, 132(12): 6091-6106.) explored the semantic correlations within and between images by decoupling multi-label features to transfer the knowledge of known labels, thereby generating pseudo-labels for unknown labels. E. Cole et al. (Cole E, Mac Aodha O, Lorieul T, et al. Multi-label learning from single positive labels[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 933-942.) proposed a simple method of treating all unknown labels as negative labels to calculate the loss of the model for training the model. Z. Ding et al. (Ding Z, Wang A, Chen H, et al. Exploring structured semantic prior for multi-label recognition with incomplete labels[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2023: 3398-3407.) improved the recognition performance of the model under missing multi-labels by using the CLIP model to learn more discriminative features. However, some of the above methods require decoupling multi-label features, but it is difficult to guarantee the quality of the decoupled features in the case of severely missing labels. On the other hand, treating unknown labels as negative labels in some of the above methods will not only introduce a large amount of label noise but also exacerbate the imbalance problem between positive and negative labels. Finally, the method of using CLIP as mentioned above will bring additional complexity and memory overhead.

[0004] In summary, the following deficiencies exist in current partial multi-label medical image recognition solutions:

[0005] (1) Low decoupling quality: In the case of severely missing labels, using the method based on label decoupling makes it difficult to guarantee the quality of decoupled features.

[0006] (2) Introduction of label noise: Due to the large imbalance between positive and negative labels in medical images, using the method of treating all unknown labels as negative labels will lead to the introduction of a large amount of label noise and further exacerbate the positive-negative imbalance problem.

[0007] (3) Additional complexity and overhead: The method based on the CLIP model brings additional computational complexity and memory overhead.

[0008] In the field of medical image recognition, partial multi-label medical image recognition is a common problem in clinical practice. However, due to the high complexity of this task, researchers usually simplify the task, and few researchers directly focus on this field. Summary of the Invention

[0009] To solve the problems in the prior art, this application provides a partial multi-label medical image recognition method based on a pseudo-label recognition strategy.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] On the one hand, the present invention provides a partial multi-label medical image recognition method based on a pseudo-label recognition strategy, including:

[0012] S1: Obtain a training data set containing multiple medical image training samples and labels. Each medical image training sample corresponds to multiple labels, and perform partial multi-label processing on the training data set; obtain multiple preset categories according to the labels of the medical image training samples.

[0013] S2: Construct an image recognition model, where the image recognition model includes an image encoder and a classifier; the image recognition model is used to first extract image features from the input image through the image encoder, and then analyze the image features through the classifier to obtain the prediction probabilities corresponding to each preset category. The prediction probability represents the probability that the corresponding preset category is the positive label of the medical image training sample.

[0014] S3: Input the partially multi-label processed training data set into the image recognition model, perform pseudo-label recognition based on the prediction probabilities output by the model, and train the image recognition model based on the recognized pseudo-labels.

[0015] S4: Use the trained image recognition model to process the medical image to be recognized, and obtain the image recognition result according to the prediction probabilities output by the model.

[0016] On the other hand, the present invention also proposes a partial multi-label medical image recognition system based on a pseudo-label recognition strategy, which is used to implement the above method.

[0017] It can be seen from the above technical solution that the advantages of the present invention are:

[0018] 1. The present invention uses an asymmetric dual threshold strategy to use a lower threshold for the identification of negative pseudo labels and a higher threshold for the identification of positive pseudo labels, thereby initially generating high-quality pseudo labels. And further, by capturing the co-occurrence relationship between labels, potential positive labels in unknown labels are further identified. These two strategies can make image recognition more accurate in medical scenarios with multiple labels missing, thereby improving the accuracy and reliability of medical diagnosis.

[0019] 2. This paper proposes a new multi-label medical image recognition method in the label-missing scenario based on asymmetric dual threshold and label conditional co-occurrence probability matrix strategy. The asymmetric dual threshold strategy uses a lower threshold for the identification of negative pseudo labels and a higher threshold for the identification of positive pseudo labels to initially generate high-quality pseudo labels; the label conditional co-occurrence probability matrix strategy uses the co-occurrence relationship of multiple labels to further mine potential positive labels in unknown labels. Through these two strategies, the model's multi-label recognition performance in the label-missing scenario is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.

[0021] Figure 1 A schematic diagram of method steps in one embodiment of the present invention;

[0022] Figure 2 Schematic diagram of the process of model training in one embodiment of the present invention;

[0023] Figure 3 A schematic diagram of the structure of an image recognition model in one embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of an input medical image in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0026] Example 1

[0027] The present invention preliminarily identifies pseudo labels by using an asymmetric dual-threshold pseudo label recognition strategy, and further uses the co-occurrence relationship between labels to mine potential positive label information. Finally, in order to avoid introducing noise labels and balance positive and negative labels, the model is trained using PASL loss. This method solves the technical problems of insufficient pseudo label recognition performance, low recognition accuracy and reliability of existing partial multi-label medical images.

[0028] refer to Figures 1 to 4 ,like Figure 1 As shown, this embodiment provides a partial multi-label medical image recognition method based on an asymmetric dual-threshold pseudo-label recognition strategy and a pseudo-label recognition strategy of a label conditional co-occurrence relationship. In this embodiment, the pseudo-label recognition strategy is based on an asymmetric dual-threshold and a label conditional co-occurrence relationship. Specifically, this method uses an asymmetric dual-threshold pseudo-label recognition strategy to use a lower threshold for the recognition of negative pseudo-labels and a higher threshold for the recognition of positive pseudo-labels, thereby initially generating high-quality pseudo-labels; and this method further identifies potential positive labels in unknown labels by capturing the co-occurrence relationship between labels; finally, this method uses PASL Loss to train the model to solve the imbalance problem between positive and negative labels, so that the image recognition accuracy is higher when labels are missing, thereby improving the accuracy and reliability of medical diagnosis.

[0029] In this embodiment, the partial multi-label medical image recognition method based on the pseudo-label recognition strategy specifically includes:

[0030] Step S1: construct a training data set, including multiple medical image samples and their labels, and perform partial multi-label processing on the labels of the training data set;

[0031] Step S2: constructing an image recognition model;

[0032] Step S3: using part of the multi-label processed training data set to train the image recognition model, the training adopts a pseudo-label recognition strategy based on an asymmetric dual-threshold pseudo-label recognition strategy and a label conditional co-occurrence relationship;

[0033] Step S4: pre-process the medical image to be recognized, input the pre-processed medical image data into a trained image recognition model, and obtain the image recognition result according to the model output.

[0034] In this embodiment, medical images such as X-ray images and magnetic resonance images, each medical image sample has multiple labels. The partial multi-label processing refers to randomly deleting a preset proportion of sample labels to simulate the situation where some labels are missing.

[0035] like Figure 3 As shown, the image recognition model includes an image encoder f and a classifier h.

[0036] Feature extraction is performed on the input image data based on the image encoder f to obtain image features, and the classifier h is used to output a prediction probability, and an image classification and recognition result is obtained based on the prediction probability.

[0037] In this embodiment, the image encoder structure adopts a Densenet encoder; the structure of the classifier is a fully connected layer. In this example, the input dimension of the fully connected layer is the same as the output feature dimension of the image encoder, and the output dimension is the number C of classification categories.

[0038] When training the image recognition model, two strategies need to be adopted, including an asymmetric double-threshold pseudo-label recognition strategy and a label conditional co-occurrence relationship pseudo-label recognition strategy. Among them, the training data set includes multiple groups of medical images and the labels corresponding to the medical images after partial multi-label processing.

[0039] Let D = {I, Y} represent a training data set with C categories and N samples, where I represents the input multi-label medical image, and Y = {y1, y2, y3,......, y C} represents the label set corresponding to each image, where the C labels correspond to the C categories in sequence, and y c ∈{-1, 0, 1} represents the label corresponding to the c-th category in the label set, where -1 represents a negative label, 0 represents an unknown label, and 1 represents a positive label.

[0040] The training process is specifically as Figure 2 shown. Training the image recognition model based on the training data set includes the following steps:

[0041] Step (1): Randomly sample n groups of medical image training samples I from the training data set, and the labels y corresponding to the training samples I after partial multi-label processing;

[0042] Step (2): Input the training samples into the image encoder f to obtain image features, and input the obtained image features into the classifier to obtain the prediction probability z. The prediction probability represents the probability that the corresponding preset category is the positive label of the medical image training sample;

[0043] Step (3): Compare the prediction probability z with the thresholds τ1, τ2 for preliminary pseudo-label recognition to obtain a preliminary pseudo-label

[0044]

[0045] Among them, represents the preliminary pseudo-label generated by the c-th category of the i-th medical image training sample based on the thresholds τ1, τ2, denotes the predicted probability value of the \(i\)-th training sample for the \(c\)-th class. \(\tau_1\) and \(\tau_2\) are thresholds for preliminary pseudo-label identification, and \(\tau_1+\tau_2\neq1\);

[0046] In this embodiment, the thresholds are \(\tau_1 = 0.2\) and \(\tau_2 = 0.9\).

[0047] Step (4): Obtain the conditional co-occurrence probability matrix \(A\) updated after the previous training. If this is the first training, calculate the conditional co-occurrence probability matrix \(A\) according to the following formula:

[0048] A=\{p j,c |j\in[1,C],c\in[1,C]\}

[0049]

[0050] where \(p j,c denotes the conditional co-occurrence probability of the \(c\)-th class and the \(j\)-th class, and \(f j,c denotes the probability of co-occurrence of the \(j\)-th class and the \(c\)-th class, that is, the number of times the corresponding labels of the \(j\)-th and \(c\)-th classes are both 1 in the current training batch. \(f j denotes the frequency of the \(j\)-th class, that is, the number of times the corresponding label of the \(j\)-th class is 1 in the current training batch. \(C\) represents the preset number of classes;

[0051] Step (5): Obtain the further pseudo-labels from the preliminarily identified pseudo-labels

[0052]

[0053] where, denotes the further pseudo-label of the \(i\)-th training sample for the \(c\)-th class, denotes the set of conditional co-occurrence probabilities of the \(c\)-th class and the current \(j\)-th class, and \(\max\{p c \}\) denotes the maximum conditional co-occurrence probability value in the set \(p c ;

[0054] In this embodiment, the thresholds are \(\tau_3 = 0.5\) and \(\tau_4 = 0.85\).

[0055] Step (6): Update the conditional probability co-occurrence matrix \(A\) in one training batch:

[0056] A′=(1 - \alpha)A+\alpha A batch

[0057]

[0058] Among them, A′ represents the updated conditional probability co-occurrence matrix in the current training batch, and A batch represents the conditional co-occurrence probability matrix calculated based on the further obtained pseudo-labels in the current training batch , and α represents a hyperparameter used to control the fusion ratio; in this embodiment, α = 0.1;

[0059] Step (7): Calculate the partial asymmetric loss function L for the predicted probability z and the further obtained pseudo-labels : PASL

[0060]

[0061] where γ + <γ - are all hyperparameters for controlling positive and negative imbalance.

[0062] For each image recognition model, calculate the forward propagation error value according to the loss function LBCE, and then perform backpropagation according to the error to optimize the model parameters to reduce the loss function value;

[0063] Step (8): Repeat the above steps until the number of iterations is greater than the iteration threshold e.

[0064] In this embodiment, e = 50.

[0065] In step S4, preprocess the medical image to be recognized, input the preprocessed medical image data into the trained image recognition model for classification and recognition, and obtain the image recognition result according to the model output.

[0066] In this embodiment, preprocessing the medical image to be recognized includes: reading a single medical image I to be recognized and performing image enhancement processing on the medical image.

[0067] The trained model recognizes the medical image to be recognized and outputs the predicted probability. For the predicted probability z corresponding to the c-th category output by the model c , if the predicted probability is greater than 0.5, the corresponding category exists; otherwise, it does not exist. The image recognition result includes all categories corresponding to the predicted probabilities greater than 0.5.

[0068] In this embodiment, the medical image as shown in Figure 4 is preprocessed and input into the trained image recognition model for classification and recognition, and the image recognition results are "cardiac enlargement", "effusion", "emphysema", "mass", and after verification, it is completely correct.

[0069] Embodiment 2

[0070] ​In this embodiment, a partial multi-label medical image recognition system based on a pseudo-label recognition strategy is also provided. This system is used to implement the above embodiment. The following terms such as "module" and "unit" can be a combination of software and / or hardware that can achieve a predetermined function. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible.

[0071] In this embodiment, the partial multi-label medical image recognition system based on the pseudo-label recognition strategy includes the following modules:

[0072] A data preprocessing module, which is used to obtain a training data set and perform partial multi-label processing, and at the same time obtain multiple preset categories;

[0073] A model construction module, which is used to construct an image recognition model including an image encoder and a classifier;

[0074] A model training module, which is used to train the image recognition model with the training data set and optimize the model parameters through pseudo-label recognition;

[0075] An image recognition module, which is used to recognize the medical image to be recognized with the trained image recognition model.

[0076] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be elaborated here. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without creative work.

[0077] The embodiments of the system of the present invention can be applied to any device with data processing capabilities. This any device with data processing capabilities can be a device or apparatus such as a computer. The system embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of any device with data processing capabilities reading the corresponding computer program instructions in the non-volatile memory into the memory for operation.

[0078] In order to perform accurate medical image recognition in the case of multiple missing labels, the present invention provides a training method based on a pseudo-label recognition strategy; by adopting an asymmetric double threshold, a lower threshold is used for the recognition of negative pseudo-labels and a higher threshold is used for the recognition of positive pseudo-labels, thereby initially generating high-quality pseudo-labels; and further identifying potential positive labels in unknown labels by capturing the co-occurrence relationship between labels; finally, in order not to introduce label noise, the PASL loss is used as the loss function for training.

[0079] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A partial multi-label medical image recognition method based on a pseudo-label recognition strategy, characterized in that, Including: S1: Obtain a training data set including multiple medical image training samples and labels. Each medical image training sample corresponds to multiple labels, and perform partial multi-label processing on the training data set; Obtain multiple preset categories according to the labels of the medical image training samples; S2: Construct an image recognition model, which includes an image encoder and a classifier. The image recognition model is used to first extract image features of the input image through the image encoder, and then analyze the image features through the classifier to obtain the prediction probabilities corresponding to each preset category. The prediction probability represents the probability that the corresponding preset category is the positive label of the medical image training sample; S3: Input the training data set after partial multi-label processing into the image recognition model, perform pseudo-label recognition based on the prediction probabilities output by the model, and train the image recognition model based on the recognized pseudo-labels; S4: Use the trained image recognition model to process the medical image to be recognized, and obtain the image recognition result according to the prediction probabilities output by the model.

2. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 1, wherein, In S1, the partial multi-label processing includes performing random masking on the sample labels at a preset ratio.

3. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 1, wherein, The medical image is an X-ray image or a nuclear magnetic resonance image.

4. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 1, wherein, S3 specifically includes multiple rounds of repeated iterative training. Each round of training includes steps 1)-7): 1) Randomly sample multiple medical image training samples and their corresponding labels after partial multi-label processing from the training data set; 2) Input the sampled training samples into the image encoder to obtain image features, and input the obtained image features into the classifier to obtain prediction probabilities. The prediction probability represents the probability that the corresponding preset category is the positive label of the medical image training sample; 3) Based on the prediction probabilities, for the labels with a value of 0 after partial multi-label processing, use the asymmetric double-threshold pseudo-label recognition strategy to compare the prediction probabilities with two preset thresholds for preliminary pseudo-label recognition to obtain preliminary pseudo-labels; 4) Obtain the conditional co-occurrence probability matrix updated in the previous round of training; If this round of training is the first round of training, calculate the conditional co-occurrence probability matrix based on the preliminary pseudo-labels; 5) Based on the prediction probabilities and the conditional co-occurrence probability matrix, use the label conditional co-occurrence relationship pseudo-label recognition strategy to obtain further pseudo-labels; 6) Update the conditional probability co-occurrence matrix based on the further pseudo-labels; 7) Calculate the partial asymmetric loss function based on the prediction probabilities and the further pseudo-labels, and optimize the model parameters to reduce the value of the loss function; Repeat steps 1)-7) until the number of iterations is greater than the threshold.

5. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 4, characterized in that, In step 3), the specific formula for using the asymmetric double-threshold pseudo-label recognition strategy to compare the prediction probabilities with two preset thresholds for preliminary pseudo-label recognition to obtain preliminary pseudo-labels is: Among them, represents the preliminary pseudo-label corresponding to the c-th category of the i-th image, indicates that the corresponding category is a negative label, indicates that the corresponding category is an unknown label, indicates that the corresponding category is a positive label, and τ1, τ2 are thresholds for preliminary pseudo-label recognition, where τ1 + τ2 ≠ 1.

6. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 4, characterized in that, In step 4), the formula for calculating the conditional co-occurrence probability matrix based on the preliminary pseudo-labels is: A = {p j,c | j ∈ [1, C], c ∈ [1, C]} Among them, A represents the conditional co-occurrence probability matrix, and p j,c represents the conditional co-occurrence probability of the c-th category and the j-th category, and f l,c represents the number of times that the preliminary pseudo-labels corresponding to the j-th category and the c-th category are both 1, and f j represents the number of times that the further pseudo-label corresponding to the j-th category is 1, and C represents the preset number of categories.

7. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 4, characterized in that, In step 5), the specific formula for using the label conditional co-occurrence relationship pseudo-label recognition strategy to obtain further pseudo-labels is: Among them, represents the further pseudo-label of the c-th category of the i-th training sample, represents the predicted probability of the c-th category of the i-th image, represents the set of conditional co-occurrence probabilities, p k,c represents the probability that the preliminary pseudo-labels corresponding to the j-th category and the c-th category are both 1, represents the preliminary pseudo-label corresponding to the c-th category of the i-th image, and max{p c} represents the maximum conditional co-occurrence probability value in the set p c τ2, τ3, and τ4 are preset thresholds.

8. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 4, characterized in that In step 6), the specific formula for updating the conditional probability co-occurrence matrix based on the further pseudo-labels is: A′ = (1 - α)A + αA batch Among them, A′ represents the updated conditional probability co-occurrence matrix, A represents the conditional probability co-occurrence matrix before update, A batch represents the conditional co-occurrence probability matrix calculated based on further pseudo-labels, α represents a hyperparameter, represents the number of times that the further pseudo-labels corresponding to the j-th category and the c-th category are both 1, represents the number of times that the further pseudo-label corresponding to the j-th category is 1, and C represents the preset number of categories.

9. The partial multi-label medical image recognition method based on the pseudo-label recognition strategy according to claim 4, wherein, In step 7), the formula for the partial asymmetric loss function is: Among them, L PASL represents the value of the partial asymmetric loss function, and γ + <γ - are all hyperparameters, represents the predicted probability of the c-th category of the i-th image, represents the further pseudo-label of the c-th category of the i-th image.

10. A partial multi-label medical image recognition system based on a pseudo-label recognition strategy, for implementing the partial multi-label medical image recognition method described in claim 1, characterized in that, Including: A data preprocessing module, which is used to obtain a training data set and perform partial multi-label processing, and at the same time obtain multiple preset categories; A model construction module, which is used to construct an image recognition model including an image encoder and a classifier; A model training module, which is used to train the image recognition model using the training data set and optimize the model parameters through pseudo-label recognition; An image recognition module, which is used to recognize the medical image to be recognized using the trained image recognition model.

Images