A target classification method, system and electronic device for a partial multi-labeled image

Abstract

The application provides a target classification method and system for a partial multi-labeled image and an electronic device, and relates to the technical field of target classification for a partial multi-labeled image. The method comprises the following steps: acquiring a to-be-identified partial multi-labeled image; inputting the to-be-identified partial multi-labeled image into a related label determination model to determine the related labels of all targets in the to-be-identified partial multi-labeled image; and determining the categories of all targets in the to-be-identified partial multi-labeled image according to the multiple related labels. The related labels correspond to the target categories one by one. The application utilizes the contrast label disambiguation principle to train a classifier to obtain a multi-label determination model capable of accurately identifying related labels in an unseen image, thereby improving the accuracy of image classification.

Description

Technical Field

[0001] This invention relates to the field of target classification technology for multi-labeled images, and in particular to a target classification method, system, and electronic device for multi-labeled images. Background Technology

[0002] Multi-label image classification aims to address the image classification problem where each image is associated with multiple labels. Currently, multi-label image classification has received widespread attention. However, it relies heavily on accurate data labeling, which is extremely difficult to achieve in resource-constrained real-world scenarios. To alleviate the labeling burden, current approaches involve assigning each image multiple candidate labels by non-professional annotators. These candidate labels include not only relevant labels beneficial for image classification but also some noise labels. The method of learning from this set of candidate labels is defined as the partial multi-label image classification problem.

[0003] Partial multi-label learning problems fall into two categories: The first involves assigning a confidence level to each candidate label and iteratively updating the label confidence levels and classification model parameters during training. For example, the paper "Partial Multi-Label Learning" proposes optimizing the label ranking confidence matrix by considering label relevance and feature prototypes separately during classifier training. The paper "Feature-Induced Partial Multi-label Learning" introduces a feature-induced partial multi-label method by considering the low-rank characteristics of the label and feature spaces. The paper "Partial Multi-Label Learning with Meta Disambiguation" proposes achieving disambiguation by iteratively minimizing the confidence-weighted ranking loss and adaptively estimating the confidence level of each candidate label using the model's performance on the validation set. The second type is a two-stage training method, which obtains reliable labels from the candidate label set and then uses these reliable labels to train a multi-label classifier. For example, the paper "Discriminative and Correlative Partial Multi-Label Learning" applies feature manifolds to induce high-confidence labels, thereby training a multi-label classifier. The paper "PartialMulti-Label Learning via Credible Label Elicitation" induces a classifier by extracting reliable labels through an iterative label propagation strategy. Regarding patents, Chinese invention patent application number 202010412162.1 provides a partial multi-label learning method based on multi-subspace representation; Chinese invention patent application number 202010412161.7 provides a partial multi-label learning method based on noise tolerance; Chinese invention patent application number 202111369388.9 provides a patient screening label method based on partial multi-label learning; Chinese invention patent application number 202010411579.6 provides a partial multi-label learning method based on global and local label relationships; Chinese invention patent application number 202110717550.5 provides a partial multi-label learning method based on complementary label collaborative training; and Chinese invention patent application number 202010411580.9 provides a partial multi-label learning method for features with noise, utilizing the ideas of low-rank and sparse decomposition to recover correct feature information and effectively reduce the influence of noisy feature information. Although these traditional methods have made significant progress, they are generally based on learning from handcrafted features. When faced with target classification problems of heavily labeled images, their representation and label correction capabilities are weak, and they cannot achieve good label disambiguation results. Summary of the Invention

[0004] The purpose of this invention is to provide a target classification method, system, and electronic device for multi-labeled images. By using the principle of contrastive label disambiguation to train a classifier, a relevant label determination model can be obtained that can accurately identify relevant labels in unseen multi-labeled images, thereby improving the accuracy of image classification.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A target classification method for multi-labeled images includes:

[0007] Obtain an image of multiple markers to be identified; the image of multiple markers to be identified includes at least one target;

[0008] The image with multiple labels to be identified is input into the relevant label determination model to determine the relevant labels of all targets in the image with multiple labels to be identified; the relevant label determination model is obtained by training a classifier based on multiple historical images with multiple labels using the principle of contrast label disambiguation;

[0009] The types of all targets in the multi-labeled image to be identified are determined based on multiple related labels; the related labels correspond one-to-one with the target types.

[0010] Optionally, before obtaining the image to be classified, the following steps are also included:

[0011] Multiple historical images with excessive labeling are acquired; each historical image with excessive labeling is marked with various labels; the types of labels are relevant labels or noise labels.

[0012] Random data augmentation is performed on the over-labeled historical images to obtain query and key views of the over-labeled historical images;

[0013] Determine the tag-level embedding under the query view and the tag-level embedding under the key view; the tag-level embedding corresponds one-to-one with multiple tags on the biased tag history image;

[0014] Based on the tag-level embeddings under the query view and the tag-level embeddings under the key view, the classifier is trained using the contrastive tag disambiguation principle to obtain the relevant tag determination model.

[0015] Optionally, after determining the tag-level embedding under the query view and the tag-level embedding under the key view, the following may also be included:

[0016] Determine the positive or negative nature of multiple tag-level embeddings under the query view;

[0017] Determine the positive or negative nature of multiple tag-level embeddings under the key view.

[0018] Optionally, the step of training the classifier based on the label-level embeddings under the query view and the label-level embeddings under the key view, using the principle of contrastive label disambiguation, to obtain the relevant label determination model includes:

[0019] The classifier is determined to be the classifier used in the 0th iteration;

[0020] The initial prototype of each label in the classifier is obtained as the prototype at the 0th iteration;

[0021] The initial negative prototype of each label in the classifier is obtained as the negative prototype at the 0th iteration;

[0022] Let the first iteration number i = 1;

[0023] Let the second iteration number j = 1;

[0024] Determine any tag-level embedding under any query-based view as the current tag-level embedding;

[0025] Update the positive prototype and the negative prototype at the (i-1)th iteration based on the positive and negative values ​​of the current tag-level embedding.

[0026] The first similarity is calculated by comparing the current tag-level embedding with the prototype at the (i-1)th iteration after the update.

[0027] The second similarity is calculated by comparing the current tag-level embedding with the negative prototype at the (i-1)th iteration after the update.

[0028] Based on the first similarity and the second similarity, determine the label vector predicted by the prototype for the current label-level embedding;

[0029] Update the pseudo-tag of the current tag-level embedding corresponding tag according to the tag vector to obtain the pseudo-tag of the current tag-level embedding corresponding tag in the j-th iteration;

[0030] Increment the value of the second iteration number j by 1, update the current tag-level embedding to a current tag-level embedding other than the current tag-level embedding under the same query view, and return to the step "update the positive prototype and the negative prototype at the (i-1)th iteration according to the positive and negative signs of the current tag-level embedding" until the second iteration number reaches the second iteration number threshold.

[0031] Embed multiple current label levels under the corresponding query view into the classifier input at the (i-1)th iteration to obtain multiple category outputs;

[0032] Based on the output of multiple categories and the pseudo-labels of the corresponding labels embedded at the current label level during multiple iterations, determine the classification loss function for the (i-1)th iteration;

[0033] Determine whether the classification loss function is less than the classification loss function threshold to obtain a first determination result;

[0034] If the first judgment result is negative, then update the parameters of the classifier at the (i-1)th iteration to obtain the classifier at the ith iteration, increment the value of the first iteration number i by 1, and return to the step "set the second iteration number j = 1";

[0035] If the first judgment result is yes, then determine whether the first iteration number has reached the first iteration number threshold, and obtain the second judgment result;

[0036] If the second judgment result is negative, then the classifier in the (i-1)th iteration is determined to be the classifier in the ith iteration, the value of the first iteration number i is increased by 1, and the process returns to the step "set the second iteration number j = 1".

[0037] If the second judgment result is yes, then the classifier in the (i-1)th iteration is determined to be the related label determination model.

[0038] Optionally, before determining the classifier for the (i-1)th iteration as the relevant label to determine the model, the method further includes:

[0039] The tag-level embeddings under the query view and the tag-level embeddings under the key view are identified as the embedding pool; the embedding pool also includes the tag-level embeddings in the momentum tag-level embedding queue.

[0040] Determine any tag-level embedding with a positive sign under the query view as the current positive tag-level embedding;

[0041] The set of positive samples corresponding to the current positive label-level embedding is determined by identifying the positive label-level embedding in the embedding pool that has the same label as the positive label-level embedding.

[0042] The current positive label-level embedding and the samples in the positive sample set corresponding to the current positive label-level embedding constitute multiple positive sample pairs;

[0043] Determine the contrastive loss function for the corresponding biased historical image based on multiple positive sample pairs under the same query view;

[0044] A third judgment result is obtained by determining whether all of the aforementioned contrast loss functions are less than the contrast loss function threshold;

[0045] If the third judgment result is negative, then update the parameters of the classifier at the (i-1)th iteration to obtain the classifier at the 0th iteration, and return to the step "set the first iteration number i = 1";

[0046] If the third judgment result is yes, then the step "determine the classifier for the (i-1)th iteration as the relevant label determination model" is called.

[0047] A target classification system for multi-labeled images includes:

[0048] A module for acquiring images with multiple markers to be identified is used to acquire images with multiple markers to be identified; the images with multiple markers to be identified include at least one target;

[0049] The relevant label recognition module is used to input the image with multiple labels to be identified into the relevant label determination model to determine the relevant labels of all targets in the image with multiple labels to be identified; the relevant label determination model is obtained by training a classifier based on multiple historical images with multiple labels using the principle of contrast label disambiguation;

[0050] The target type determination module is used to determine the type of all targets in the multi-marker image to be identified based on multiple related markers; the related markers correspond one-to-one with the target types.

[0051] An electronic device, optionally, includes a memory and a processor, the memory for storing a computer program, the processor running the computer program to cause the electronic device to perform the target classification method for a partially labeled image.

[0052] Optionally, the memory is a readable storage medium.

[0053] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0054] This invention provides a method, system, and electronic device for target classification in multi-labeled images. The method involves acquiring a multi-labeled image to be identified; inputting the image into a correlation label determination model to determine the correlation labels of all targets in the image; determining the types of all targets in the image based on multiple correlation labels; and ensuring a one-to-one correspondence between correlation labels and target types. This invention utilizes the principle of contrastive label disambiguation to train a classifier, resulting in a correlation label determination model capable of accurately identifying correlation labels in unseen images, thereby improving the accuracy of image classification. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1This is a flowchart of a target classification method for partially labeled images according to Embodiment 1 of the present invention;

[0057] Figure 2 This is a CPLD model framework diagram according to Embodiment 2 of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] The purpose of this invention is to provide a target classification method, system, and electronic device for multi-labeled images. By using the principle of contrastive label disambiguation to train a classifier, a relevant label determination model can be obtained that can accurately identify relevant labels in unseen multi-labeled images, thereby improving the accuracy of image classification.

[0060] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] Example 1

[0062] like Figure 1 As shown, this embodiment provides a target classification method for multi-labeled images, including:

[0063] Step 101: Obtain the image with multiple markers to be identified.

[0064] Among them, the image to be identified with multiple labels must contain at least one target.

[0065] Step 102: Input the image with multiple labels to be identified into the relevant label determination model to determine the relevant labels of all targets in the image with multiple labels to be identified.

[0066] The relevant label determination model is obtained by training a classifier based on multiple historical images with multiple labels and using the principle of contrastive label disambiguation.

[0067] Step 103: Determine the types of all targets in the multi-labeled image to be identified based on multiple related labels; the related labels correspond one-to-one with the target types.

[0068] Before step 101, the following is also included:

[0069] Step 104: Obtain multiple historical images with multiple labels; these images contain various labels. The labels can be classified as relevant or noise labels.

[0070] Step 105: Perform random data augmentation on the over-labeled historical images to obtain the query view and key view of the over-labeled historical images.

[0071] Step 106: Determine the tag-level embeddings under the query view and the key view. The tag-level embeddings correspond one-to-one with multiple tags on the multi-tag history image.

[0072] Step 107: Based on the label-level embeddings under the query view and the label-level embeddings under the key view, train the classifier using the contrastive label disambiguation principle to obtain the relevant label determination model.

[0073] Following step 106, the following is also included:

[0074] Step 108: Determine the positive or negative nature of multiple tag-level embeddings under the query view.

[0075] Step 109: Determine the positive or negative of multiple tag-level embeddings under the key view.

[0076] Step 107 includes:

[0077] Step 1071: Determine the classifier as the classifier used in the 0th iteration.

[0078] Step 1072: Obtain the initial prototype of each label in the classifier as the prototype at the 0th iteration.

[0079] Step 1073: Obtain the initial negative prototype of each label in the classifier as the negative prototype at the 0th iteration.

[0080] Step 1074: Let the first iteration number i = 1.

[0081] Step 1075: Let the second iteration number j = 1.

[0082] Step 1076: Determine any tag-level embedding under any query-based view as the current tag-level embedding.

[0083] Step 1077: Update the positive prototype and the negative prototype at the (i-1)th iteration based on the positive and negative values ​​of the current tag-level embedding.

[0084] Step 1078: Calculate the similarity between the current tag-level embedding and the prototype at the (i-1)th iteration after the update as the first similarity.

[0085] Step 1079: Calculate the similarity between the current tag-level embedding and the negative prototype at the (i-1)th iteration after the update as the second similarity.

[0086] Step 10710: Determine the label vector predicted by the prototype for the current label-level embedding based on the first similarity and the second similarity.

[0087] Step 10711: Update the pseudo-tag of the current tag-level embedding corresponding to the tag based on the tag vector, and obtain the pseudo-tag of the current tag-level embedding corresponding to the tag in the j-th iteration.

[0088] Step 10712: Increment the value of the second iteration number j by 1, update the current tag-level embedding to a current tag-level embedding other than the current tag-level embedding under the same query view, and return to step 1077 until the second iteration number reaches the second iteration number threshold.

[0089] Step 10713: Embed the multiple current label levels under the corresponding query view into the classifier at the (i-1)th iteration to obtain multiple category outputs.

[0090] Step 10714: Determine the classification loss function for the (i-1)th iteration based on the output of multiple categories and the pseudo-labels of the corresponding labels embedded at the current label level during multiple iterations.

[0091] Step 10715: Determine whether the classification loss function is less than the classification loss function threshold to obtain the first judgment result; if the first judgment result is no, proceed to step 10716; if the first judgment result is yes, proceed to step 10717.

[0092] Step 10716: Update the parameters of the classifier in the (i-1)th iteration to obtain the classifier in the i-th iteration, increment the value of the first iteration number i by 1, and return to step 1075.

[0093] Step 10717: Determine whether the number of the first iterations has reached the threshold for the number of the first iterations, and obtain the second determination result; if the second determination result is negative, proceed to step 10718; if the second determination result is positive, proceed to step 10719.

[0094] Step 10718: Determine the classifier for the (i-1)th iteration as the classifier for the i-th iteration, increment the value of the first iteration number i by 1, and return to step 1075.

[0095] Step 10719: Determine the classifier for the (i-1)th iteration as the relevant label determination model.

[0096] Before step 10719, the following is also included:

[0097] Step 10720: Determine the tag-level embeddings under the query view and the tag-level embeddings under the key view as the embedding pool; the embedding pool also includes the tag-level embeddings in the momentum tag-level embedding queue.

[0098] Step 10721: Determine any tag-level embedding with a positive sign under the query view as the current positive tag-level embedding.

[0099] Step 10722: Determine the set of positive samples corresponding to the current positive label-level embedding in the embedding pool that has the same label as the positive label-level embedding.

[0100] Step 10723: Determine that the current positive label-level embedding and the samples in the positive sample set corresponding to the current positive label-level embedding constitute multiple positive sample pairs.

[0101] Step 10724: Determine the contrastive loss function for the corresponding biased historical image based on multiple positive sample pairs under the same query view.

[0102] Step 10725: Determine whether all the contrast loss functions are less than the contrast loss function threshold to obtain the third judgment result; if the third judgment result is no, proceed to step 10726; if the third judgment result is yes, proceed to step 10727.

[0103] Step 10726: Update the parameters of the classifier in the (i-1)th iteration to obtain the classifier in the 0th iteration, and return to step 1074.

[0104] Step 10727: If the result of the third judgment is yes, then call step 10719.

[0105] Example 2

[0106] like Figure 2 As shown, the target classification method for multi-labeled images provided in this embodiment consists of two parts: a contrastive learning module and a prototype-based label disambiguation module. This method uses these two modules to construct a collaborative system framework: contrastive learning aims to obtain high-quality representations, while prototype-based label disambiguation utilizes the high-quality representations learned through contrastive learning to obtain improved prototypes, which are then used to update pseudo-labels and guide the model's prediction results, thus helping contrastive learning establish more accurate positive sample pairs. Simultaneously, this invention employs a two-stage training strategy to make contrastive learning technology more rationally applied to this invention. The two modules are interdependent and work together; as training progresses, the model can gradually update the confidence of the labels, extract relevant labels, and reduce attention to noisy labels. The specific steps are as follows:

[0107] Step 1: Use predominantly labeled image training data as input. Definition and These are the feature space and the label space, respectively, where K represents the number of labels of interest. Training dataset. It consists of n samples, where This represents the i-th observed image. Let y represent the candidate label vector corresponding to the i-th image. i,j =1 indicates that label j is the label of the i-th image, and vice versa.

[0108] Step 2: Obtain an augmented view of the image:

[0109] For the sake of brevity, index i is omitted in this invention. For the input image x, this invention uses two image augmentation methods to obtain the query view Aug. q (x) and key view Aug k (x).

[0110] Specifically, for the query network, this invention uses the SimAugment data augmentation method from the paper "Supervised contrastive learning", and for the key network, this invention uses the RandAugment data augmentation method from the paper "RandAugment: Practical Automated Data Augmentation With a Reduced Search Space".

[0111] Step 3: Obtain tag-level embeddings in both views:

[0112] For the query view, Enc(Aug) is obtained through its encoder network. q (x)∈R d*h*w Where d, h, and w represent the dimension of the encoder output, the height, and the width of the feature map, respectively. The feature map is then reduced to K dimensionality using a 1*1 convolution to obtain the feature map corresponding to each class. The class feature maps are flattened into vector form and input into a projection head to project the class vectors onto the required contrast space. This yields the output g(Aug) of the query network g(·). q (x)∈R K *D Where D is the dimension of the contrast space. Therefore, the image-level feature map is decoupled into K D-dimensional label-level embeddings q. j ∈R 1*D For each label-level embedding, j∈{1,...,K}, it can be viewed as a representation vector of the image against the corresponding label background, containing feature information of the corresponding class. The key network g'(·) is the result of the moving average of the parameter momentum of the query network, and the obtained g'(Aug k (x)∈R K*D To prepare for subsequent use of comparative learning, each line k j ∈R 1*Dj∈{1,...,K} and q j Similarly, this indicates that the key network decouples the image-level representation to obtain the label-level embedding.

[0113] Step 4: Obtain high-quality embedding representations using contrastive learning:

[0114] Step 1: Determine the positive or negative nature of the tag-level embedding.

[0115] For the augmented image Aug mentioned above q (x), after passing through the classifier Obtain the output f(Aug) of the classifier q (x))∈[0,1 K , where f(Aug q Each term of (x) is the output of the Sigmoid activation function.

[0116] use Determine the positive or negative value of each tag-level embedding. Where, if Then, determine if the j-th label-level embedding is positive, and vice versa. Aggregate these label-level embeddings to obtain the set of positive / negative label-level embeddings of the sample image, PE(x) / NE(x). In the above formula, α is a hyperparameter, b j and These are the baseline probability of class j and the average baseline probability for all classes, respectively.

[0117] Step 2: Construct a set of positive samples for each positive label-level embedding.

[0118] The positive label embeddings obtained from the query network and the key network are combined with the historical embeddings in the embedding queue to construct an embedding pool A = B. q ∪B k ∪queue, and A(q) j )=A\{q j}, where B q and B k This indicates that all sample images in the current batch correspond to the positive label-level embeddings of the query network and the key network. This invention uses a queue to retain the positive label-level embeddings of the most recent batch of samples obtained from the key network.

[0119] Use P(q) j )={k j |k j ∈A j (q j )} indicates positive tag-level embedding q j The set of positive samples, where means A(q jThe above formula shows that the set of positive samples for each positive label-level embedding is the set of other positive label-level embeddings of the same category in the embedding pool.

[0120] Step 3: Establish comparative loss.

[0121] By constructing positive sample pairs with all positive label-level embeddings and samples in their positive sample sets, a contrastive loss is established to obtain high-quality label-level embedding representations.

[0122] The formula for calculating the contrast loss function of a single image sample is as follows:

[0123]

[0124] Where τ represents the temperature parameter.

[0125] Step 5: Prototype-based label disambiguation:

[0126] For each class c∈{1,...,K}, this invention uses a prototype. and a negative prototype These represent the representative positive / negative label-level embedding features of class c.

[0127] Step 1: Update the positive / negative prototype.

[0128] In the current mini-batch, the positive / negative prototypes of the corresponding classes are updated using the positive / negative label-level embeddings of the samples. The update formula is as follows:

[0129]

[0130] Step 2: Update the pseudo-tags.

[0131] The similarity between the label-level embedding of the sample and the improved positive / negative prototype is calculated to obtain the label vector z predicted by the prototype. The pseudo-labels s are then updated incrementally using a moving average method.

[0132]

[0133] Where φ∈(0,1) are normal values. The above equation indicates that if a tag's label-level embedding q... c After calculating the similarity with the corresponding positive / negative prototypes, it is found that the positive prototype is more similar, so this label is considered as the relevant label for image x. As the model training progresses, the predictions of the label-level embeddings obtained through the corresponding prototypes will gradually become consistent. Therefore, the pseudo-labels of relevant labels will gradually stabilize at 1, while the pseudo-labels of irrelevant labels will smoothly approach 0.

[0134] Step 3: Construct a classification loss using the classifier's output and pseudo-labels.

[0135] The classification loss is established using the updated pseudo-labels and the classifier's output. Contrastive learning is used to improve the prototype, and the improved pseudo-labels guide the model's predictions. The classification loss is calculated as follows:

[0136]

[0137] Step 6: Combine the classification loss and contrastive loss to form the total loss function, which serves as the objective function for training and optimizing the neural network. The total loss function is calculated as follows:

[0138]

[0139] Here, λ is an adjustable hyperparameter.

[0140] Step 7: Use a two-stage training strategy:

[0141] The two-stage training strategy consists of a pre-disambiguation stage and a contrastive disambiguation stage.

[0142] Pre-disambiguation stage: The contrastive learning branch is removed, meaning only the query network and prototype disambiguation strategy are used. At this stage, only... As the objective function of the network.

[0143] Contrast disambiguation phase: The entire system is trained using the total loss function.

[0144] Step 8: Use a threshold δ to perform multi-label prediction for unseen samples on the test data:

[0145] For an unseen sample x, its associated label prediction result is:

[0146]

[0147] For any unseen instance x, the classifier's predicted probability is compared to a threshold δ to determine whether the label is relevant. A multi-label evaluation metric is then used to evaluate the model.

[0148] This invention was tested on the mainstream multi-label image classification dataset VOC2007. The VOC2007 dataset contains images from 20 target categories, with each image containing an average of 2.5 target categories. The VOC2007 dataset includes a training dataset of 5011 images and a test dataset of 4952 images.

[0149] To construct a predominantly labeled dataset, this invention sets the average size of the label set to be q, which is the proportion of the total number of labels in the label space. In the experiment, q was set to 0.1 for the VOC2007 dataset.

[0150] Following many existing studies, this invention uses mAP, OF1, and CF1 as evaluation metrics. mAP, also known as the average precision across all classes, is a weighted average of the average accuracy (AP) across all classes. CF1 and OF1 comprehensively consider both overall recall and precision for each class. Therefore, these three metrics are the most important and representative evaluation metrics among all metrics.

[0151] To verify the effectiveness of this invention, a baseline method is established by directly training the binary cross-entropy loss (BCE) on a partially labeled image dataset. Additionally, two advanced multi-label image classification methods, ASL (published in "Asymmetric loss for multi-label classification") and ML-GCN (published in "Multi-Label Image Recognition with Graph Convolutional Networks"), are added as two other benchmark comparison methods.

[0152] To ensure consistency in comparisons, this invention follows the research of ASL and uses TresnetL (Tresnet: High performance GPU-dedicated architecture), pre-trained on ImageNet, as its backbone, with an input image resolution of 224*224. For the VOC2007 dataset, α = 0.8. The prototype is updated using γ = 0.99, and the constant φ for pseudo-label updates is linearly decreased from 0.95 to 0.8. The temperature parameter τ = 0.2, the weight factor λ of the loss function = 0.1, and the model parameters are updated using an exponential moving average (EMA) with a decay parameter of 0.9997.

[0153] The method proposed in this invention is named CPLD (Contrastive Prototype-based Label Disambiguation), and the comparison results are shown in Table 1. It can be seen that on the VOC2007 dataset, the method of this invention outperforms other methods, demonstrating the effectiveness of the proposed method.

[0154] Table 1 Comparison of Model Classification Results

[0155] Methods / Indicators mAP CF1 OF1 BCE 88.37 82.17 84.42 ML-GCN 80.31 68.13 72.81 ASL 87.88 79.78 81.62 CPLD 89.79 82.68 85.33

[0156] This invention provides a target classification method for partially labeled images. It directly uses partially labeled images to train data and obtains a classification model that can predict multiple labels for unseen instances, greatly reducing labeling costs.

[0157] Furthermore, the classification model in this invention uses the binary cross-entropy loss function (BCE). More advanced loss calculation methods such as FocalLoss (Focal Loss for Dense Object Detection) or ASL (Asymmetric Loss for Multi-Label Classification) can also be used. Additionally, for determining the positive or negative of the label-level embeddings obtained from model decoupling, a simple fixed threshold can be applied to the classifier's predicted probabilities, or the predicted probabilities can be obtained using softmax and then... As a threshold, these improvements or other easily conceivable variations or substitutions are all within the scope of protection of this invention.

[0158] Example 3

[0159] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a target classification system for multi-labeled images is provided below, including:

[0160] The module for acquiring images with multiple markers to be identified is used to acquire images with multiple markers to be identified; the images with multiple markers to be identified include at least one target.

[0161] The relevant label recognition module is used to input the image with multiple labels to be identified into the relevant label determination model to determine the relevant labels of all targets in the image with multiple labels to be identified. The relevant label determination model is trained on the classifier based on multiple historical images with multiple labels and using the principle of contrast label disambiguation.

[0162] The target category determination module is used to determine the category of all targets in the multi-labeled image to be identified based on multiple related labels; the related labels correspond one-to-one with the target categories.

[0163] Example 4

[0164] This embodiment provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform a target classification method for partially labeled images as described in Embodiment 1. The memory is a readable storage medium.

[0165] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0166] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A target classification method for multi-labeled images, characterized in that, include: Obtain the image with the most markers to be identified; The image to be identified with multiple markers includes at least one target; The image with multiple markers to be identified is input into the relevant marker determination model to determine the relevant markers of all targets in the image with multiple markers to be identified; The relevant label determination model is obtained by training a classifier based on multiple historical images with multiple labels and using the principle of contrastive label disambiguation. Before acquiring the image to be classified, the process further includes: acquiring multiple historical images with multiple labels; the historical images with multiple labels are labeled with various labels; the types of labels are relevant labels or noise labels; performing random data augmentation on the historical images with multiple labels to obtain a query view and a key view of the historical images with multiple labels; determining the label-level embeddings under the query view and the label-level embeddings under the key view; the label-level embeddings correspond one-to-one with the multiple labels on the historical images with multiple labels; training the classifier using the contrastive label disambiguation principle based on the label embeddings under the query view and the label embeddings under the key view to obtain the relevant label determination model; inputting the multiple current label-level embeddings under the corresponding query view into the classifier at the (i-1)th iteration to obtain multiple category outputs; and determining the classification model based on the multiple category outputs and the current label embeddings at the key view during multiple iterations. The pseudo-labels corresponding to the pre-label level are embedded to determine the classification loss function at the (i-1)th iteration. It is then determined whether the classification loss function is less than a classification loss function threshold to obtain a first judgment result. If the first judgment result is negative, the parameters of the classifier at the (i-1)th iteration are updated to obtain the classifier at the ith iteration. The value of the first iteration number i is increased by 1, and the process returns to the step "set the second iteration number j=1". If the first judgment result is positive, it is then determined whether the first iteration number reaches the first iteration number threshold to obtain a second judgment result. If the second judgment result is negative, the classifier at the (i-1)th iteration is determined to be the classifier at the ith iteration. The value of the first iteration number i is increased by 1, and the process returns to the step "set the second iteration number j=1". If the second judgment result is positive, the classifier at the (i-1)th iteration is determined to be the relevant label determination model. The types of all targets in the multi-labeled image to be identified are determined based on multiple related labels; the related labels correspond one-to-one with the target types.

2. The target classification method for multi-labeled images according to claim 1, characterized in that, After determining the tag-level embedding under the query view and the tag-level embedding under the key view, the following is also included: Determine the positive or negative nature of multiple tag-level embeddings under the query view; Determine the positive or negative nature of multiple tag-level embeddings under the key view.

3. The target classification method for partially labeled images according to claim 2, characterized in that, The process of training a classifier based on the label-level embeddings under the query view and the label-level embeddings under the key view, using the principle of contrastive label disambiguation, to obtain the relevant label determination model includes: The classifier is determined to be the classifier used in the 0th iteration; The initial prototype of each label in the classifier is obtained as the prototype at the 0th iteration; The initial negative prototype of each label in the classifier is obtained as the negative prototype at the 0th iteration; Let the first iteration number i = 1; Let the second iteration number j=1; Determine any tag-level embedding under any query-based view as the current tag-level embedding; Update the positive prototype and the negative prototype at the (i-1)th iteration based on the positive and negative values ​​of the current tag-level embedding. The first similarity is calculated by comparing the current tag-level embedding with the prototype at the (i-1)th iteration after the update. The second similarity is calculated by comparing the current tag-level embedding with the negative prototype at the (i-1)th iteration after the update. Based on the first similarity and the second similarity, determine the label vector predicted by the prototype for the current label-level embedding; Update the pseudo-tag of the current tag-level embedding corresponding tag according to the tag vector to obtain the pseudo-tag of the current tag-level embedding corresponding tag in the j-th iteration; Increment the value of the second iteration number j by 1, update the current tag-level embedding to a current tag-level embedding other than the current tag-level embedding under the same query view, and return to the step "update the positive prototype and the negative prototype at the (i-1)th iteration according to the positive and negative of the current tag-level embedding" until the second iteration number reaches the second iteration number threshold.

4. The target classification method for multi-labeled images according to claim 2, characterized in that, Before determining the classifier for the (i-1)th iteration as the model for the relevant labels, the process further includes: The tag-level embeddings under the query view and the tag-level embeddings under the key view are identified as the embedding pool; the embedding pool also includes the tag-level embeddings in the momentum tag-level embedding queue. Determine any tag-level embedding with a positive sign under the query view as the current positive tag-level embedding; The set of positive samples corresponding to the current positive label-level embedding is determined by identifying the positive label-level embedding in the embedding pool that has the same label as the positive label-level embedding. The current positive label-level embedding and the samples in the positive sample set corresponding to the current positive label-level embedding constitute multiple positive sample pairs; Determine the contrastive loss function for the corresponding biased historical image based on multiple positive sample pairs under the same query view; A third judgment result is obtained by determining whether all of the aforementioned contrast loss functions are less than the contrast loss function threshold; If the third judgment result is negative, then update the parameters of the classifier at the (i-1)th iteration to obtain the classifier at the 0th iteration, and return to the step "set the first iteration number i=1"; If the third judgment result is yes, then the step "determine the classifier for the (i-1)th iteration as the relevant label determination model" is called.

5. A target classification system for multi-labeled images, characterized in that, The target classification system for the partially labeled image uses the target classification method for partially labeled images as described in any one of claims 1-4, wherein the partially labeled image target classification system comprises: A module for acquiring images with multiple markers to be identified is used to acquire images with multiple markers to be identified; the images with multiple markers to be identified include at least one target; The relevant label recognition module is used to input the image with multiple labels to be identified into the relevant label determination model to determine the relevant labels of all targets in the image with multiple labels to be identified; the relevant label determination model is obtained by training a classifier based on multiple historical images with multiple labels using the principle of contrast label disambiguation; The target type determination module is used to determine the type of all targets in the multi-marker image to be identified based on multiple related markers; the related markers correspond one-to-one with the target types.

6. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform a target classification method for a multi-labeled image according to any one of claims 1 to 4.

7. An electronic device according to claim 6, characterized in that, The memory is a readable storage medium.

Images