Multi-Label Image Classification With Visual-Semantic Contrastive Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-label image classification technologies suffer from misidentification and omission of identification labels, leading to low accuracy in assigning multiple category labels to images.
Innovation Solution
A multi-label image classification model training method that utilizes a visual encoder and a contrastive learning network to acquire visual and semantic features, constructs correlation matrices, and adjusts parameters based on multiple loss functions to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing image classification technologies classify images merely based on content, then the classification process is simple, but misidentification and omission of identification labels easily occur, resulting in low accuracy
Solution Approach 1:
The patent combines visual features extracted from image content with semantic features extracted from label sets. The contrastive learning network integrates these two feature types to learn their correlations, merging content-based classification with semantics-based classification to improve overall accuracy while managing model complexity through feature fusion.
Solution Approach 2:
The patent introduces a contrastive learning network as an intermediary component between the visual encoder and the classification output. This intermediary learns the correlation between visual features and semantic features, acting as a bridge that connects content-based representation with label-based representation to reduce misidentification and omission.
2Measurement precision
If multiple loss functions are used to train the model, then the model convergence accuracy is improved, but the training process becomes more complex
Solution Approach 1:
The patent employs multiple loss functions (contrastive loss, classification loss, and regularization loss) with different parameter optimization objectives. By changing the optimization parameters through diverse loss functions, the model achieves better convergence accuracy, with each loss function targeting specific aspects of model performance.
Solution Approach 2:
The training framework uses multiple loss functions that serve different purposes simultaneously: contrastive loss for feature alignment, classification loss for label prediction accuracy, and regularization loss for preventing overfitting. This multi-functional approach to loss functions comprehensively improves model performance while managing training complexity through a unified optimization framework.
Data Source
Figure 1
Figure 2~4
Figure 5~6
AI summary
Disclosed in the present invention are a multi-label image classification method and apparatus, and a multi-label image classification model training method and apparatus. The model training method comprises: acquiring training samples; acquiring semantic features of label sets; inputting sample images and the semantic features into a multi-label image classification model, acquiring visual features from the sample images, obtaining the correlations between the sample images and the label sets on the basis of a comparison learning network, and on the basis of a fully-connected network, obtaining prediction results of the sample images mapped to classification labels; constructing a correlation matrix and a unit matrix; and calculating target loss on the basis of the correlation matrix, the unit matrix, the prediction results, and the corresponding classification labels, and adjusting parameters of the multi-label image classification model until a convergence condition is met. According to the present invention, semantic information in labels is utilized, and in addition to training of prediction results of the labels corresponding to image information, the correlation between the image information and the semantic information is further learned, so that a high-accuracy multi-label image classification model is obtained, thereby improving the accuracy of multi-label image classification.