Methods for Constructing Instance-Level Classifiers Based on Weakly Supervised Self-Training

Abstract

This invention discloses a method for constructing an instance-level classifier based on weakly supervised self-training, comprising the following steps: Step 1: Constructing an instance-level classifier by obtaining different types of instance datasets, dividing the instance datasets into positive packet instance datasets and negative packet instance datasets, labeling the positive packet instance datasets to obtain pseudo-labels, and labeling the negative packet instance datasets to obtain true negative labels; Step 2: Inputting the instance datasets into the instance-level classifier to obtain prediction results, and training the instance-level classifier based on the prediction results, the pseudo-labels, and the true negative labels; Step 3: Iterating through steps 1 and 2 until the pseudo-labels and the parameters of the instance-level classifier converge, thereby obtaining the trained instance-level classifier. This invention can classify packets and also classify instances within packets.

Description

Technical Field

[0001] This invention belongs to the field of multi-instance learning technology in machine learning, and particularly relates to a method for constructing instance-level classifiers based on weakly supervised self-training. Background Technology

[0002] Multiple instance learning (MIL) is a weakly supervised learning problem encountered in many fields, such as drug design, pathological image classification, and video anomaly detection. In a typical MIL scenario, multiple instances form a bag whose labels are known and used for training, while the labels of the instances are unknown. Traditional MIL research has mainly focused on bag classification tasks, but instance-level classification is also receiving increasing attention because it helps improve the interpretability of classification models and discover new knowledge.

[0003] Since only bag labels are available for training, existing mainstream methods address this problem from the perspective of bag classification. This bag classification method aggregates the features of all instances in a bag to obtain bag-level features and trains a bag-level classifier to classify them. In recent years, research in this direction has focused on developing more efficient aggregation functions by using attention, graph convolutional networks (GCN), and Transformers. However, this bag classification method compromises instance-level classification ability because the loss function is mainly based on the bag classification results, ignoring some important instance-level information. In the instance feature space, even if the bag-level classifier can only distinguish one simple positive instance, it can still correctly classify the bag, lacking the motivation for further optimization. This leads to two problems: (1) the model may not be able to distinguish positive instances, thus the instance-level classification ability is insufficient. (2) if all positive training bags have some simple positive instances, the trained model may not be able to generalize well to bags with only positive instances. In addition, theoretical analysis of bag classification methods from the perspective of information theory shows that no matter what aggregation method is used, this method will lose a lot of information by ignoring negative instances. Summary of the Invention

[0004] The purpose of this invention is to propose a method for constructing an instance-level classifier based on weakly supervised self-training, which can classify packets more accurately and classify instances within each packet.

[0005] To achieve the above objectives, this invention provides a method for constructing an instance-level classifier based on weakly supervised self-training, comprising the following steps:

[0006] Step 1: Construct an instance-level classifier, obtain different types of instance datasets, divide the instance datasets into positive packet instance datasets and negative packet instance datasets, label the positive packet instance datasets to obtain pseudo-labels, and label the negative packet instance datasets to obtain true negative labels;

[0007] Step 2: Input the instance dataset into the instance-level classifier to obtain the prediction results, and train the instance-level classifier based on the prediction results, the pseudo-labels, and the true negative labels;

[0008] Step 3: Iterate through Step 1 and Step 2 until the pseudo-labels and the parameters of the instance-level classifier converge, and obtain the trained instance-level classifier.

[0009] Optionally, the instance dataset includes: feature vectors obtained by extracting features from image data or raw data; if the data input to the instance-level classifier is image data, then the image data includes an image encoding network and a category prediction head network; if the data input to the instance-level classifier is a feature vector obtained by extracting features from raw data, then the features include a prediction head network.

[0010] Optionally, labeling the positive packet instance dataset to obtain pseudo-labels specifically includes:

[0011]

[0012]

[0013] In the formula, <·> represents the Frobenius dot product, P is the original network output, Q is the pseudo label, U(r,c) represents the transport matrix with edge distributions of r and c, and the parameter μ represents the proportion of positive instances in all positive packets.

[0014] Optionally, the parameter μ can be set using a dynamic transformation strategy, including:

[0015]

[0016] Where, μ t The value of parameter μ represents the value of the parameter in the t-th training round, where T is the value of μ. t The number of cycles that converge to μ.

[0017] Optionally, inputting the instance dataset into the instance-level classifier to obtain prediction results, and training the instance-level classifier based on the prediction results, the pseudo-labels, and the true negative labels specifically includes:

[0018] Calculate the first cross-entropy between the output parameters of the positive packet instance dataset in the instance-level classifier and the pseudo-label, calculate the second cross-entropy between the output parameters of the negative packet instance dataset in the instance set classifier and the true negative label, optimize the first cross-entropy and the second cross-entropy using gradient descent, and obtain the minimum cross-entropy of the parameters of the instance-level classifier.

[0019] Optionally, gradient descent is used to optimize the first cross-entropy and the second cross-entropy to obtain the minimum cross-entropy of the instance-level classifier, wherein the minimum cross-entropy of the network parameters is expressed as:

[0020]

[0021] Where θ is the parameter of classifier f, CE is the cross-entropy function, and x n For instances from the negative packet, y n For true negative labels, x p For instances from positive packets, q p This is a pseudo-tag.

[0022] Optionally, the convergence process of the pseudo-labels includes: optimizing the pseudo-labels using a fast Sinkhorn-Knopp algorithm, calculated as follows:

[0023] First, add a regular expression to the optimization objective:

[0024]

[0025] Where KL represents the Kullback-Leibler divergence, and when λ is large, the solution of the function of the optimization objective is similar to the original expression. The solutions are similar;

[0026] After adding a regularization term, the optimization problem has a solution:

[0027] Q = diag(α)P λ diag(β)

[0028] In the formula, α and β are scaling coefficients, which can be obtained through iteration:

[0029]

[0030] Where P is the original output of the instance-level classification network, and y and i are the number of iterations.

[0031] Optionally, the convergence of the parameters of the instance-level classifier is achieved when the parameters of the instance-level classifier obtain the minimum cross-entropy.

[0032] Technical advantages of this invention: This invention discloses a method for constructing an instance-level classifier based on weakly supervised self-training. It directly trains an instance-level classifier that can classify not only bags but also instances within bags. Compared with traditional attention-based methods, it has more accurate instance classification performance and better interpretability. The bag classification capability of this invention is stronger than traditional methods, surpassing existing methods on multiple international public datasets. This invention has strong versatility and is applicable to all multi-instance learning scenarios, including pathological image processing, video abnormal frame detection, drug analysis, etc. Attached Figure Description

[0033] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0034] Figure 1 This is a flowchart illustrating the method for constructing an instance-level classifier based on weakly supervised self-training according to an embodiment of the present invention. Detailed Implementation

[0035] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0037] like Figure 1 As shown, this embodiment provides a method for constructing an instance-level classifier based on weakly supervised self-training, including the following steps:

[0038] Step 1: Construct an instance-level classifier, obtain different types of instance datasets, divide the instance datasets into positive packet instance datasets and negative packet instance datasets, label the positive packet instance datasets to obtain pseudo-labels, and label the negative packet instance datasets to obtain true negative labels;

[0039] Step 2: Input the instance dataset into the instance-level classifier to obtain the prediction results, and train the instance-level classifier based on the prediction results, the pseudo-labels, and the true negative labels;

[0040] Step 3: Iterate through Step 1 and Step 2 until the pseudo-labels and the parameters of the instance-level classifier converge, and obtain the trained instance-level classifier.

[0041] Initialize the instance classification network, which includes building an instance-level classifier f. If the input data is image data, its structure includes an image encoding network and a class prediction head network. If the input data is features, its structure only contains a prediction head network. Then, the instances in the positive packet are... The input is fed into the instance classifier to obtain the prediction result.

[0042]

[0043] All instances from the positive packets are input into the instance classifier to obtain the prediction results for these instances. Each row of P represents an instance, and the two columns represent the probability that the instance is positive or negative.

[0044] To prevent P from degenerating, new pseudo-labels are then constructed based on it. And to make it satisfy a specific edge distribution, the specific implementation method is to solve an optimal transmission problem, namely:

[0045]

[0046]

[0047] In the formula, <·> represents the Frobenius dot product, U(r, c) represents the transfer matrix with marginal distributions of r and c, μ represents the proportion of positive instances in all positive packets, and the solution method adopts the fast Sinkhorn-Knopp algorithm, that is, first adding a regularization term to the optimization objective:

[0048]

[0049] Where KL represents the Kullback-Leibler divergence, and when λ is large, the solution to the optimization problem is similar to the solution to the original problem. After adding a regularization term, the optimization problem has a solution:

[0050] Q = diag(α)P λ diag(β)

[0051] In the formula, α and β are scaling coefficients, which can be obtained through iteration:

[0052]

[0053] Where P is the original output of the instance-level classification network, and y and i are the number of iterations.

[0054] After network initialization, all instances are input into the instance classifier. For instances from positive packets, the cross-entropy between the classifier's prediction and the pseudo-label obtained in step 1 is calculated. For instances from negative packets, the cross-entropy between the classifier's prediction and the true negative label is calculated. The classifier is then trained to minimize these two cross-entropies.

[0055]

[0056] Where θ is the parameter of classifier f, CE is the cross-entropy function, and x n For instances from the negative packet, y n For true negative labels, x p For instances from positive packets, q p This is a pseudo-tag.

[0057] The pseudo-label generation and classifier training are alternately iterated until the pseudo-label and classification networks converge. To avoid selecting a local optimum at the beginning of the convergence process, a dynamic transformation strategy is adopted for the parameter μ setting in the pseudo-label generation step.

[0058]

[0059] In the formula μ t The value of parameter μ represents the value of the parameter in the t-th training round, where T is the value of μ. t The number of epochs converges to μ. Thus, during the initial training, μ... t A value close to 0.5 causes the network to generate a large number of false positive labels; as training progresses, μ is then increased... t It converges to μ.

[0060] The test data to be predicted is input into the trained instance classification network. First, the prediction results of all instances are obtained. Then, the instance prediction results in each bag are aggregated by max pooling to obtain the prediction result of the bag.

[0061]

[0062] in, x represents the network's prediction of packets. i，k Given an example image as input, f(x) i，k ) is the network for instance x i，k The prediction results.

[0063] The method of this invention is used to analyze breast pathological slide images.

[0064] This invention employs a weakly supervised and self-training method for end-to-end training, and the specific implementation process is as follows:

[0065] Step 1: Divide the 400 whole pathological images in the CAMELYON16 dataset into a training set of 270 cases and a test set of 130 cases. Then, segment each whole image using a non-overlapping segmentation method, dividing it into image blocks of size 512x512. Each image block is then considered an instance, and all image blocks from the same pathological image are grouped into a single package.

[0066] Step 2: Randomly initialize an instance classifier. The classifier's encoder is a ResNet18, the classification head network is a fully connected neural network, the number of input neurons is 512, and the number of output neurons is 2. Input all instances from positive packets into this classifier to obtain the prediction result P for all instances. Then, construct pseudo-labels Q according to the method in Step 2 of the invention, setting the parameter μ to 0.85 during construction.

[0067] Step 3: Input all instances into the instance classifier, calculate the loss function using the constructed pseudo-labels and true negative labels, and use the Adam algorithm to optimize the classifier parameters to minimize the loss function.

[0068] Step 4: Alternately repeat steps 2 and 3, using a dynamic parameter μ. In the early stages of iteration, μ is close to 0.5, and as training progresses, μ is gradually made closer to the set value of 0.85. Then, μ is maintained at 0.85 until the pseudo-labels and network parameters converge.

[0069] Step 5: Apply the trained network to the test dataset. Input the instances in the test packets into the trained network to obtain the prediction results for each instance in each packet. Calculate the AUC (Advanced Value Comparison) of the prediction results with the real instance labels. Then, use a max-pooling strategy to aggregate the prediction results of all instances in each packet to obtain the packet prediction results. Calculate the AUC of this result with the real packet labels. The results are shown in Table 1.

[0070] Table 1

[0071]

[0072] The experimental results of this embodiment show that the present invention achieves better accuracy in both instance classification and package classification tasks, surpassing other existing methods; moreover, the present invention is highly versatile, simple to implement, and can be used in various scenarios that conform to multi-instance learning, including pathological image analysis.

[0073] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing an instance-level classifier based on weakly supervised self-training, characterized in that, The method comprises the following steps: Step 1, constructing an instance-level classifier, obtaining different types of instance datasets, the instance datasets being image data, dividing the instance datasets into positive package instance datasets and negative package instance datasets, marking the positive package instance datasets to obtain pseudo labels, and marking the negative package instance datasets to obtain true negative labels; The pseudo labels are obtained by marking the positive package instance datasets, and the marking specifically comprises: , wherein, in the formula represents the Frobenius point multiplication, is the original output of the instance-level classification network, is the pseudo label, represents the edge distribution is and the transmission matrix, the parameter represents the proportion of positive instances in all positive bags; Adopting dynamic transformation strategy to parameter Setting includes: wherein, representative parameters in the first value at the time of training, is converge to the epoch; Step 2, inputting the instance datasets into the instance-level classifier to obtain a prediction result, and training the instance-level classifier based on the prediction result, the pseudo labels and the true negative labels; Step 3, iterating steps 1 and 2 until the pseudo labels and parameters of the instance-level classifier converge, and obtaining a trained instance-level classifier.

2. The weakly supervised self-training based instance-level classifier construction method of claim 1, wherein, The instance datasets comprise image data or feature vectors obtained after extracting features from original data; if the data input into the instance-level classifier is image data, the image data comprises an image encoding network and a category prediction head network; if the data input into the instance-level classifier is feature vectors obtained after extracting features from original data, the features comprise a prediction head network. 3.The weakly supervised self-training based instance-level classifier construction method of claim 1, wherein, The prediction result is obtained by inputting the instance datasets into the instance-level classifier, and the instance-level classifier is trained based on the prediction result, the pseudo labels and the true negative labels, and the training specifically comprises: The first cross-entropy of the output parameters of the positive package instance datasets in the instance-level classifier and the pseudo labels is calculated, the second cross-entropy of the output parameters of the negative package instance datasets in the instance-level classifier and the true negative labels is calculated, the first cross-entropy and the second cross-entropy are optimized by using a gradient descent method, and the parameter minimum cross-entropy of the instance-level classifier is obtained.

4. The weakly supervised self-training based instance-level classifier construction method of claim 3, wherein, The first cross-entropy and the second cross-entropy are optimized by using a gradient descent method, and the parameter minimum cross-entropy of the instance-level classifier is obtained, wherein the network parameter minimum cross-entropy is expressed as: wherein, is a parameter of the classifier , is a cross-entropy function, is an instance from a negative bag, is a true negative label, is an instance from a positive bag, is a pseudo label.

5. The weakly supervised self-training based instance-level classifier construction method of claim 1, wherein, The convergence process of the pseudo labels comprises: based on the pseudo labels, a fast Sinkhorn-Knopp algorithm is used to optimize the pseudo labels, and the following calculation is performed: First, a regularization term is added to the optimization target: wherein represents the Kullback-Leibler divergence, when in the formula is large, the solution of the function of the optimization objective is similar to the solution of the original formula ; is a pseudo-label; and are marginal distributions; is the original output of the instance-level classification network; After adding the regularization term, the solution for optimizing the pseudo labels is expressed as: wherein and are scaling factors, obtained by iteration: , wherein, is the original output of the instance-level classification network, and is the number of iterations.

6. The weakly supervised self-training based instance-level classifier construction method of claim 1, wherein, The convergence of the parameters of the instance-level classifier is that the parameters of the instance-level classifier obtain the minimum cross-entropy.

Images