A semi-supervised multi-label image classification method, system, device and storage medium

Abstract

The application discloses a kind of semi-supervised multi-label image classification method, system, equipment and storage medium.The method utilizes feature extraction network to obtain the spatial feature representation of input image, and respectively constructs class-independent global feature and class-specific global attention feature;Feature map is divided into local area, local average feature and local class-specific attention feature are generated, the fusion of global feature and local feature is realized by residual form, attention enhancement prediction of teacher network is constructed based on fusion feature, and the optimal threshold value of each class is searched on labeled data using class adaptive threshold estimation method, to generate high-quality pseudo label on unlabeled data;Semi-supervised collaborative optimization is realized by combining network training again.The application can effectively improve the accuracy and stability of multi-label image classification under limited annotation conditions, and is especially suitable for medical image recognition, retail commodity detection, remote sensing scene analysis and other multi-label complex scenes.

Description

Technical Field

[0001] This invention relates to the field of computer technology applications, specifically to a semi-supervised multi-label image classification method, system, device, and storage medium. Background Technology

[0002] With the rapid development of deep learning in computer vision, multi-label image classification has become a core task in scenarios such as intelligent security, medical image interpretation, product recognition, remote sensing monitoring, and industrial inspection. Unlike traditional single-label classification, multi-label classification problems often involve multiple visual semantic elements simultaneously. For example, a street scene image may contain multiple categories such as "vehicles," "pedestrians," and "traffic signs"; a medical chest X-ray may be labeled with disease tags such as "enlarged heart," "disordered lung texture," and "pleural effusion." Due to the significant co-occurrence relationships and semantic dependencies between the label semantics, multi-label classification tasks are usually more challenging.

[0003] Most existing multi-label learning methods rely on a large amount of manually labeled data to ensure sufficient supervision signals, but manually labeled multi-label images have the following prominent difficulties:

[0004] Labeling is costly and time-consuming. Multi-label images typically involve dozens or even hundreds of categories, requiring annotators to simultaneously determine the presence of multiple targets, significantly increasing labeling costs. This is especially true in specialized fields such as medicine, remote sensing, and industrial inspection, where the professional skills required of annotators are extremely high, making it difficult to obtain large-scale, high-quality labeled data.

[0005] The categories are extremely imbalanced. In real-world data, most categories exhibit a long-tailed distribution, with a minority of categories appearing with extremely low frequency. Conventional supervised learning struggles to learn effective features from these categories.

[0006] The complex inter-class correlations make threshold selection difficult. Multi-label predictions output by the model usually need to be binarized by a threshold, but the distribution of different classes varies greatly. Fixed threshold strategies are difficult to balance the minority and majority classes, often leading to false positives and false negatives.

[0007] Existing semi-supervised methods are heavily reliant on pseudo-labels, leading to severe noise accumulation problems. Typical semi-supervised frameworks, such as fixed-match and mean-teacher, typically generate pseudo-labels for unlabeled data and use them directly for training. When the teacher network's confidence in certain categories is unstable, it is prone to passing erroneous pseudo-labels to the student network, further amplifying training noise and causing model performance degradation or even training failure.

[0008] Existing methods generally neglect the construction of "class-specific representations". Most methods rely solely on the global representation of the backbone network for classification, making it difficult to capture the spatial regions or local structures that are most discriminative for a specific class, especially in image tasks containing small objects, dense structures, or local anomalies.

[0009] Based on the aforementioned shortcomings of existing technologies, there is an urgent need for a new semi-supervised multi-label image classification technology that can simultaneously integrate global and local visual features, automatically focus on key spatial regions based on category features, automatically learn the optimal pseudo-label threshold for each category, and effectively avoid the accumulation of pseudo-label noise in semi-supervised applications. This would solve the problems of insufficient robustness, unstable performance, and strong dependence on annotation in the practical application of current technologies. Summary of the Invention

[0010] Purpose of the invention: To address the problems of insufficient reliability of pseudo-labels caused by fixed thresholds, noise accumulation, and large inter-class differences in existing technologies, this invention provides a semi-supervised classification method for multi-label image tasks. This method overcomes the difficulty of locating key regions in traditional methods, improves the accuracy and stability of pseudo-label generation, and significantly enhances the performance and generalization ability of multi-label classification under the condition of limited labeled data, thus meeting the needs of real-world industrial applications.

[0011] Another object of the present invention is to provide a semi-supervised multi-label image classification system, as well as an electronic device, computer storage medium and computer program product capable of implementing the above method.

[0012] Technical solution: To achieve the above objectives, the present invention adopts the following technical solution:

[0013] A semi-supervised multi-label image classification method includes the following steps:

[0014] S1, given a labeled dataset Compared with unlabeled datasets , and Two datasets of different sizes, each containing an input image. The corresponding multi-label vector is , and Given the height and width of the input image, K is the total number of categories, index i represents the sample number, and index k represents the category number. A feature extraction network is used. Feature extraction is performed on the input image to obtain the feature map. C, H, and W represent the number of feature map channels, height, and width, respectively. These are the parameters of the feature extraction network;

[0015] S2, unfold the feature map into Calculate the global class-independent average feature According to the classifier weight vector Calculate global category relevance score Using temperature parameters The softmax method yields the global attention weights. This leads to the acquisition of class-specific global attention features. Thus, global class-specific residual features are constructed. , This is the residual adjustment coefficient;

[0016] S3, divide the feature map into P non-overlapping local regions, and extract local feature vectors for each region. Calculate the local class-independent average features According to the classifier weight vector Calculate local category correlation scores and using temperature parameters The softmax method yields the local attention weights. This leads to the acquisition of class-specific local attention features. Thus, local class-specific residual features are constructed. ;

[0017] S4. The global class-specific residual features and the local class-specific residual features are fused to obtain the final class-specific feature representation. , The fusion coefficient is based on the classifier weight vector. Computational attention-enhanced output And obtain the prediction probability through the activation function. , Indicates the activation function;

[0018] S5, for each category k, search for the optimal threshold within the specified interval. , This represents the true label value of the i-th labeled sample in the k-th category. Indicates an indicator function; For the threshold, This represents the F1 score, used to measure performance within a given threshold. The model's predictive performance on class k under given conditions; and the generation of pseudo-labels for unlabeled data based on the optimal threshold. ;

[0019] S6. The generated pseudo-labels and real labels are used together to train the student network. The student network parameters are jointly optimized by supervised loss, unsupervised loss and consistency loss, and the teacher network parameters are updated to obtain the trained model. The trained model is then used for the unlabeled multi-label image classification task.

[0020] Preferably, in step S1, before extracting features, the input image is preprocessed, and the preprocessing includes one or more of the following: image size scaling to a uniform size, standardization, and image enhancement operations.

[0021] After feature extraction, layer normalization or feature standardization is applied to the feature map. Perform lightweight regularization processing.

[0022] Preferably, in step S2, the classifier weight vector matrix It is obtained through joint training in an end-to-end manner and shares gradient updates with the feature extraction network.

[0023] Preferably, in step S4, the local region is divided using one or more of the following methods: fixed-size grid division, adaptive block region, and learnable image blocks generated by additional convolutional layers or attention modules.

[0024] Preferably, in step S6, the joint loss function for training the student network is as follows:

[0025] ;

[0026] in, To monitor losses, For unsupervised loss, binary cross-entropy loss is used. This is an attention consistency loss used to constrain the consistency of attention responses for the same image under different data augmentation conditions. and These are the weighting coefficients for the corresponding loss terms.

[0027] Preferably, in step S6, the teacher network parameters are updated using an exponential moving average: ,in The attenuation coefficient is... For teacher network parameters, For student network parameters.

[0028] A semi-supervised multi-label image classification system includes:

[0029] The feature extraction module is used to extract features from labeled datasets. and unlabeled datasets Employing a feature extraction network Feature extraction is performed on the input image to obtain the feature map. ,in and Two datasets of different sizes, each with a different input image. The corresponding multi-label vector is , and Here, K represents the height and width of the input image, K represents the total number of categories, index i represents the sample number, index k represents the category number, and C, H, and W represent the number of feature map channels, height, and width, respectively. These are the parameters of the feature extraction network;

[0030] The global attention module is used to expand the feature map into Calculate the global class-independent average feature According to the classifier weight vector Calculate global category relevance score Using temperature parameters The softmax method yields the global attention weights. This leads to the acquisition of class-specific global attention features. Thus, global class-specific residual features are constructed. , This is the residual adjustment coefficient;

[0031] The local attention module is used to divide the feature map into P non-overlapping local regions and extract local feature vectors for each region. Calculate the local class-independent average features According to the classifier weight vector Calculate local category correlation scores and using temperature parameters The softmax method yields the local attention weights. This leads to the acquisition of class-specific local attention features. Thus, local class-specific residual features are constructed. ;

[0032] The teacher inference module is used to fuse global class-specific residual features with local class-specific residual features to obtain the final class-specific feature representation. , The fusion coefficient is based on the classifier weight vector. Computational attention-enhanced output And obtain the prediction probability through the activation function. , Indicates the activation function;

[0033] The pseudo-label generation module is used to search for the optimal threshold within a specified interval for each category k. , This represents the true label value of the i-th labeled sample in the k-th category. Indicates an indicator function; For the threshold, This represents the F1 score, used to measure performance within a given threshold. The model's predictive performance on class k under given conditions; and the generation of pseudo-labels for unlabeled data based on the optimal threshold. ;

[0034] The model training and application module is used to train the student network by combining the generated pseudo-labels with the real labels. It optimizes the student network parameters by jointly using supervised loss, unsupervised loss, and consistency loss, and updates the teacher network parameters to obtain the trained model. The trained model is then used for the unlabeled multi-label image classification task.

[0035] The present invention also provides an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the semi-supervised multi-label image classification method as described above.

[0036] The present invention also provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the semi-supervised multi-label image classification method as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the semi-supervised multi-label image classification method as described above.

[0038] Compared with the prior art, the present invention has the following significant advantages:

[0039] (1) A global-local decoupled class-specific attention mechanism is proposed to achieve accurate localization of salient regions, solving the problem that traditional multi-label classification methods, which are mostly based on global average features, cannot accurately perceive the key regions of a certain category. This invention introduces a class-specific global attention, class-specific local attention, and residual feature fusion structure in a semi-supervised framework for the first time, enabling the model to independently construct the most discriminative features for each category. Experiments show that this structure significantly improves the recognition ability of small target categories, rare categories, and locally salient categories.

[0040] (2) A category-adaptive threshold learning method based on F1 optimization is proposed to solve the problem that fixed thresholds or manually set thresholds are difficult to achieve ideal results in multi-label scenarios. This invention constructs a category-adaptive multi-label thresholding method that automatically searches for the optimal threshold for each category, directly aims to maximize the F1 score on the validation set, matches the actual prediction distribution of the teacher network, effectively reduces the propagation of pseudo-label errors, and improves the accuracy and stability of pseudo-label generation.

[0041] (3) Combining the teacher-student framework, the predictive consistency and training stability of the unlabeled learning stage are improved through the exponential moving average (EMA) update mechanism. This invention can be directly applied to image classification production environments and has strong engineering feasibility and deployability. Attached Figure Description

[0042] Figure 1 This is a flowchart of a semi-supervised multi-label image classification method provided in an embodiment of the present invention.

[0043] Figure 2 This is a model framework diagram of the semi-supervised multi-label image classification method provided in the embodiments of the present invention.

[0044] Figure 3 This is a structural block diagram of the semi-supervised multi-label image classification system provided in the embodiments of the present invention. Detailed Implementation

[0045] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0046] Figure 1 This is a flowchart of the semi-supervised multi-label image classification method provided in an embodiment of the present invention. Figure 2 This is a model framework diagram of the semi-supervised multi-label image classification method provided in this embodiment of the invention. (Refer to...) Figure 1 and Figure 2 The method in this embodiment mainly includes the following steps: data preparation and feature extraction initialization, global class-specific attention feature construction, local branch feature construction and fusion, teacher network attention enhancement prediction, class adaptive threshold estimation, and pseudo-label generation. This process is designed for scenarios where image class labeled data is scarce but unlabeled data is sufficient. The goal is to improve multi-label classification performance under limited labeling conditions by learning a global-local decoupled attention mechanism and a best-performing class adaptive threshold estimation method. The specific method is as follows:

[0047] Step S1: Input data preparation and feature extraction process initialization.

[0048] This embodiment first constructs a semi-supervised dataset, in which labeled data is typically small (accounting for 5% to 20% of the total data), while unlabeled data constitutes a large proportion. Assuming a labeled dataset is given... Compared with unlabeled datasets Each image Its corresponding multi-label vector The total number of categories is K, where index i represents the sample number and index k represents the category number. and The height and width of the input image.

[0049] To ensure the stability of subsequent training, the following preprocessing operations are preferably performed on the input image: (1) The image size is scaled to a uniform size (e.g., or (2) Perform standardization processing, including mean and variance normalization; (3) Optional image enhancement, such as random flipping, color perturbation, random cropping, etc., to improve the model's generalization ability.

[0050] Subsequently, the preprocessed image is input into a feature extraction network backbone, such as ResNet, SwinTransformer, or ConvNeXt, which are not limited in this invention. Preferably, a backbone containing a multi-scale structure is used to extract finer-grained spatial information. The feature map is calculated by the backbone, and is represented as follows:

[0051]

[0052] Where C, H, and W represent the number of feature map channels, height, and width, respectively;

[0053] From an engineering implementation perspective, to improve the numerical stability of subsequent attention branches, layer normalization or feature whitening can be used. Lightweight regularization is applied to enhance the consistency of expression across different spatial locations.

[0054] Furthermore, the spatial dimension is flattened as follows: .

[0055] This invention sets a classifier weight vector for each category k. The weights of all categories are stacked into a matrix. .

[0056] Step S2: Global branch feature construction and engineering implementation.

[0057] Flatten the feature map as After considering each spatial location, calculate the average feature of the globally independent classes. :

[0058]

[0059] The feasibility of this step lies in its requirement for no additional parameters and extremely low computational cost, as it can be directly implemented using the averaging operation within a deep learning framework. To construct class-specific attention, a classifier matrix is ​​used. The response values ​​for all categories at S locations are generated simultaneously using matrix inner product. The score for each spatial location with respect to category k is: And utilizing temperature parameters The attention weights are obtained using softmax:

[0060]

[0061] This process is parallelized, highly scalable, and can run efficiently on GPUs / TPUs. The softmax operation of the attention weights can control the concentration of the attention distribution by adjusting the temperature parameter T. In practice, T is set to... Within this range, parameter tuning can make the attention more focused on key response regions. Ultimately, the global class-specific attention feature is achieved through the following formula:

[0062]

[0063] To enhance the expressive power of the model, this embodiment proposes to construct global features using a residual fusion approach:

[0064]

[0065] Where parameters Can be set to The specific value can be selected based on the performance of the validation set. This residual design allows the model to appropriately introduce class-related enhancement features without disrupting the global semantic structure. Empirical evidence shows that this structure can effectively improve the minority class recognition capability.

[0066] Step S3: Local branch construction and actual implementation.

[0067] To address the challenges of highly salient local targets and severe background noise interference in industrial settings, and to enhance the identification of fine-grained local regions, this embodiment constructs local branches to supplement the shortcomings of global branches.

[0068] First, the feature map is divided into P non-overlapping local regions, and local features are obtained for each region. Local feature extraction is preferably performed using one of the following three methods:

[0069] (1) Fixed-block average pooling based on feature maps: The feature map is divided into P image patches, for example or Then, average pooling is performed on the features within each patch to obtain local features. .

[0070] (2) Adaptive block region construction based on feature map: According to the spatial response intensity, saliency distribution or predefined adaptive rules of the feature map, the feature map is divided into non-uniform regions to obtain multiple local regions of variable size or position, and the features in each local region are aggregated to obtain local feature representation. This method does not require the introduction of additional learnable parameters and can dynamically adjust the structure of local regions based on the content of the input image.

[0071] (3) Learnable local feature generation based on convolutional layers or attention modules: By introducing additional shallow convolutional structures or attention modules, local pattern modeling is performed on the original feature map to directly generate a fixed number of local feature vectors, i.e., local features. In this method, the spatial coverage and response location of local features are automatically learned by the network parameters, making it suitable for application scenarios with complex structures or prominent local targets.

[0072] Next, calculate the local class-independent average feature:

[0073]

[0074] Subsequently, similar to a global branch, a category score is constructed for each local feature:

[0075]

[0076] And use softmax to obtain the local attention weights:

[0077]

[0078] This leads to the following class-specific local attention features:

[0079]

[0080] Then, residual fusion is performed with the local average features to obtain local class-specific residual features:

[0081]

[0082] Local branches can effectively capture important information about small targets and edge structures, and are especially suitable for the following scenarios: (1) in retail scenarios, it is necessary to identify local product labels, trademarks or damaged marks; (2) in medical images, it is necessary to identify local abnormal structures (such as small nodules and microbleeds); (3) in remote sensing tasks, it is necessary to capture high-frequency local targets such as roads and vehicles.

[0083] Step S4: Class-Specific Fusion Feature Construction and Attention-Enhanced Classification. This invention fuses global and local residual features to obtain the final class-specific feature representation:

[0084] ,

[0085] Wherein the fusion coefficient , Automatic searching can be performed using a validation set, typically taking... Subsequently, attention-enhanced predictions are obtained through a linear classifier:

[0086]

[0087] in, The value logit represents the teacher network's enhanced attention prediction for sample i in category k.

[0088] Mapping the above logit to the probability space, and obtaining the predicted probability using the sigmoid function:

[0089]

[0090] The output can be directly used for supervised training on labeled data, pseudo-label generation on unlabeled data, and adaptive learning of thresholds. In practical deployments, the classifier has a very small number of parameters, resulting in almost no additional overhead on inference performance, making it suitable for mobile and real-time systems.

[0091] Step S5: Class-adaptive threshold estimation and pseudo-label generation.

[0092] This invention employs a Category-wise Adaptive Multi-label Thresholding (CAMT) method with practical application advantages to automatically determine the optimal discrimination threshold for different categories in semi-supervised multi-label classification scenarios. Specifically, CAMT does not rely on a fixed threshold but dynamically selects the optimal threshold for each category based on the model's actual performance on labeled data. For each category k, this invention searches the threshold space [0,1] with a fixed step size to maximize the F1 score on the validation set, thus obtaining:

[0093]

[0094] in, This represents the true label value of the i-th labeled sample in the k-th category. When the sample When it belongs to category k, ,otherwise ; This represents an indicator function, used to convert the predicted probabilities output by the model into binary decision results. It takes the value 1 when the condition in parentheses is met, and 0 otherwise. This represents the F1 score, used to measure performance within a given threshold. The predictive performance of the model on class k under given conditions is the harmonic average of precision and recall, defined as:

[0095] .

[0096] Optimal threshold The optimal search step size is 0.005–0.02 to balance search accuracy and computational efficiency.

[0097] The mechanism has the following practical advantages: (1) It can automatically adapt to class imbalance, and the rare class will generally match the lower optimal threshold; (2) It can dynamically compensate for the deviation of the teacher network in the output distribution of different classes; (3) It can significantly improve the accuracy of pseudo-labels in semi-supervised scenarios and reduce the cumulative effect of erroneous pseudo-labels.

[0098] For unlabeled data, this invention utilizes an adaptive optimal threshold to generate high-quality pseudo-labels:

[0099]

[0100] The pseudo-labels are generated solely by the teacher network and are not directly affected by noise from the student network, making the training process more stable.

[0101] The generated pseudo-labels are used together with the real labels for training the student network. Supervised loss, unsupervised loss, and consistency loss are used to jointly optimize the student network parameters, while the teacher network parameters are updated using an exponential moving average (EMA) mechanism. (Classifier weight matrix) It can be obtained through joint training in an end-to-end manner and can share gradient updates with the backbone network.

[0102] In summary, the following sequence should be followed in each training cycle:

[0103] (1) Compute attention-enhancing logits on labeled data using a teacher network;

[0104] (2) Estimate the optimal threshold for each class using the CAMT mechanism;

[0105] (3) Use the teacher network to generate pseudo-labels on unlabeled data;

[0106] (4) Use the student network to train on both labeled and pseudo-labeled data simultaneously;

[0107] (5) Update teacher network weights using EMA (exponential moving average).

[0108] The above steps can be implemented using standard deep learning frameworks (such as PyTorch, TensorFlow, MindSpore, etc.), without additional hardware requirements, and have strong feasibility and repeatability.

[0109] The student network employs the following joint loss:

[0110]

[0111] in To monitor the binary cross-entropy loss, For pseudo-label binary cross-entropy loss, This refers to the attention consistency loss under varying enhancement levels of the same image. To constrain the consistency of attention response for the same image under different data augmentation conditions, it may take one of the following forms, including but not limited to: L2 loss based on Euclidean distance, consistency loss based on Kullback-Leibler divergence, or feature consistency loss based on cosine similarity.

[0112] Teacher network parameters are updated using an exponential moving average:

[0113]

[0114] in The EMA attenuation coefficient is... For teacher network parameters, For student network parameters.

[0115] This invention was tested on multiple datasets and, compared to ordinary BCE supervised models, can improve mean average precision (mAP) by 10% to 20% when the annotation ratio is less than 10%. Especially for rare classes or classes with high local visual saliency, the local branching and threshold adaptation mechanisms of this invention show significant advantages.

[0116] A comparative experiment was conducted using this method and a convolutional neural network (CNN) model with BCE loss. The mAP of different methods on different label ratios P ​​of a dataset (VOC) is shown in Table 1.

[0117] Table 1. Comparison of mAP of different methods on different label ratios of the dataset.

[0118] method P=0.05 P=0.10 P=0.15 P=0.2 CNN+BCE 67.95 75.35 78.19 79.38 Method of the present invention 80.08 85.50 86.61 87.26

[0119] As can be seen from Table 1, the method proposed in this invention significantly improves the mAP index compared to the CNN model on different label ratios of a dataset, demonstrating the effectiveness and practicality of the method described in this invention.

[0120] Through the above embodiments, those skilled in the art will understand that the method of the present invention has a clear structure, well-defined steps, and strong operability, and can be directly integrated into existing semi-supervised learning frameworks. In practical applications, when the system receives an unlabeled input image, it first performs preprocessing on the image and inputs it into the trained model for feature extraction. Corresponding category discrimination features are constructed through global class-specific attention branches and local class-specific attention branches, respectively. Subsequently, the predicted probability outputs for each category are obtained through feature fusion and a linear classifier. Finally, based on the category adaptive multi-label thresholding method (CAMT), the system performs category-by-category threshold determination on the predicted probabilities, thereby converting the continuous probability outputs into corresponding multi-label annotation results, realizing automatic annotation and classification of unlabeled images.

[0121] Based on the same technical concept as the method embodiment, another embodiment of the present invention provides a semi-supervised multi-label image classification system.

[0122] The feature extraction module is used to extract features from labeled datasets. and unlabeled datasets Employing a feature extraction network Feature extraction is performed on the input image to obtain the feature map. ,in and Two datasets of different sizes, each with a different input image. The corresponding multi-label vector is , and Here, K represents the height and width of the input image, K represents the total number of categories, index i represents the sample number, index k represents the category number, and C, H, and W represent the number of feature map channels, height, and width, respectively. These are the parameters of the feature extraction network;

[0123] The global attention module is used to expand the feature map into Calculate the global class-independent average feature According to the classifier weight vector Calculate global category relevance score Using temperature parameters The softmax method yields the global attention weights. This leads to the acquisition of class-specific global attention features. Thus, global class-specific residual features are constructed. , This is the residual adjustment coefficient;

[0124] The local attention module is used to divide the feature map into P non-overlapping local regions and extract local feature vectors for each region. Calculate the local class-independent average features According to the classifier weight vector Calculate local category correlation scores and using temperature parameters The softmax method yields the local attention weights. This leads to the acquisition of class-specific local attention features. Thus, local class-specific residual features are constructed. ;

[0125] The teacher inference module is used to fuse global class-specific residual features with local class-specific residual features to obtain the final class-specific feature representation. , The fusion coefficient is based on the classifier weight vector. Computational attention-enhanced output And obtain the prediction probability through the activation function. , Indicates the activation function;

[0126] The pseudo-label generation module is used to search for the optimal threshold within a specified interval for each category k. , This represents the true label value of the i-th labeled sample in the k-th category. Indicates an indicator function; For the threshold, This represents the F1 score, used to measure performance within a given threshold. The model's predictive performance on class k under given conditions; and the generation of pseudo-labels for unlabeled data based on the optimal threshold. ;

[0127] The model training and application module is used to train the student network by combining the generated pseudo-labels with the real labels. It optimizes the student network parameters by jointly using supervised loss, unsupervised loss, and consistency loss, and updates the teacher network parameters to obtain the trained model. The trained model is then used for the unlabeled multi-label image classification task.

[0128] Another embodiment of the present invention provides an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the semi-supervised multi-label image classification method as described above.

[0129] See Figure 3 , Figure 3 This is a hardware structure block diagram of the electronic device provided in an embodiment of the present invention. For example... Figure 3 As shown, the electronic device 400 of this embodiment includes: an input device 401, a processor 402, a memory 403, a storage device 404, an optional network interface 405, a display device 406, etc.

[0130] The memory 403 stores a computer program. When the processor 402 executes the program, it calls the method steps (S1~S5) of the above embodiments, including but not limited to: data loading, warm-up training, pseudo-label generation, joint training and termination output.

[0131] The processor 402 can be a CPU, DSP, ASIC, FPGA, or other programmable logic device; the memory 404 can be RAM / ROM / flash memory / SSD, etc. The network interface 401 is used to interact with external servers or storage resources to load data or synchronize models.

[0132] The above hardware configuration is merely an example, and the specific combination and topology do not constitute a limitation on the present invention.

[0133] The present invention also provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the semi-supervised multi-label image classification method as described above.

[0134] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, electronic devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 Means for a function specified in one or more processes. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that are implemented in the process. Figure 1 The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. A semi-supervised multi-label image classification method, characterized in that, Includes the following steps: S1, given a labeled dataset Compared with unlabeled datasets , and Two datasets of different sizes, each containing an input image. The corresponding multi-label vector is , and Given the height and width of the input image, K is the total number of categories, index i represents the sample number, and index k represents the category number. A feature extraction network is used. Feature extraction is performed on the input image to obtain the feature map. C, H, and W represent the number of feature map channels, height, and width, respectively. These are the parameters of the feature extraction network; S2, unfold the feature map into Calculate the global class-independent average feature According to the classifier weight vector Calculate global category relevance score Using temperature parameters The softmax method yields the global attention weights. This leads to the acquisition of class-specific global attention features. Thus, global class-specific residual features are constructed. , This is the residual adjustment coefficient; S3, divide the feature map into P non-overlapping local regions, and extract local feature vectors for each region. Calculate the local class-independent average features According to the classifier weight vector Calculate local category correlation scores and using temperature parameters The softmax method yields the local attention weights. This leads to the acquisition of class-specific local attention features. Thus, local class-specific residual features are constructed. ; S4. The global class-specific residual features and the local class-specific residual features are fused to obtain the final class-specific feature representation. , The fusion coefficient is based on the classifier weight vector. Computational attention-enhanced output And obtain the prediction probability through the activation function. , Indicates the activation function; S5, for each category k, search for the optimal threshold within the specified interval. , This represents the true label value of the i-th labeled sample in the k-th category. Indicates an indicator function; For the threshold, This represents the F1 score, used to measure performance within a given threshold. The model's predictive performance on category k under given conditions; and the generation of pseudo-labels for unlabeled data based on the optimal threshold. ; S6. The generated pseudo-labels and real labels are used together to train the student network. The student network parameters are jointly optimized by supervised loss, unsupervised loss and consistency loss, and the teacher network parameters are updated to obtain the trained model. The trained model is then used for the unlabeled multi-label image classification task.

2. The method according to claim 1, characterized in that, In step S1, before feature extraction, the input image is preprocessed. The preprocessing includes one or more of the following: image size scaling to a uniform size, standardization, and image enhancement. After feature extraction, layer normalization or feature standardization is applied to the feature map. Perform lightweight regularization processing.

3. The method according to claim 1, characterized in that, In step S2, the classifier weight vector matrix It is obtained through joint training in an end-to-end manner and shares gradient updates with the feature extraction network.

4. The method according to claim 1, characterized in that, In step S4, the local region is divided using one or more of the following methods: fixed-size grid division, adaptive block region, and learnable image patches generated by additional convolutional layers or attention modules.

5. The method according to claim 1, characterized in that, In step S6, the joint loss function for training the student network is as follows: ； in, To monitor losses, For unsupervised loss, binary cross-entropy loss is used. This is an attention consistency loss used to constrain the consistency of attention responses for the same image under different data augmentation conditions. and These are the weighting coefficients for the corresponding loss terms.

6. The method according to claim 1, characterized in that, In step S6, the teacher network parameters are updated using an exponential moving average: ,in The attenuation coefficient is... For teacher network parameters, For student network parameters.

7. A semi-supervised multi-label image classification system, characterized in that, include: The feature extraction module is used to extract features from labeled datasets. and unlabeled datasets Employing a feature extraction network Feature extraction is performed on the input image to obtain the feature map. ,in and Two datasets of different sizes, each with a different input image. The corresponding multi-label vector is , and Here, K represents the height and width of the input image, K represents the total number of categories, index i represents the sample number, index k represents the category number, and C, H, and W represent the number of feature map channels, height, and width, respectively. These are the parameters of the feature extraction network; The global attention module is used to expand the feature map into Calculate the global class-independent average feature According to the classifier weight vector Calculate global category relevance score Using temperature parameters The softmax method yields the global attention weights. This leads to the acquisition of class-specific global attention features. Thus, global class-specific residual features are constructed. , This is the residual adjustment coefficient; The local attention module is used to divide the feature map into P non-overlapping local regions and extract local feature vectors for each region. Calculate the local class-independent average features According to the classifier weight vector Calculate local category correlation scores and using temperature parameters The softmax method yields the local attention weights. This leads to the acquisition of class-specific local attention features. Thus, local class-specific residual features are constructed. ; The teacher inference module is used to fuse global class-specific residual features with local class-specific residual features to obtain the final class-specific feature representation. , The fusion coefficient is based on the classifier weight vector. Computational attention-enhanced output And obtain the prediction probability through the activation function. , Indicates the activation function; The pseudo-label generation module is used to search for the optimal threshold within a specified interval for each category k. , This represents the true label value of the i-th labeled sample in the k-th category. Indicates an indicator function; For the threshold, This represents the F1 score, used to measure performance within a given threshold. The model's predictive performance on category k under given conditions; and the generation of pseudo-labels for unlabeled data based on the optimal threshold. ; The model training and application module is used to train the student network by combining the generated pseudo-labels with the real labels. It optimizes the student network parameters by jointly using supervised loss, unsupervised loss, and consistency loss, and updates the teacher network parameters to obtain the trained model. The trained model is then used for the unlabeled multi-label image classification task.

8. An electronic device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the semi-supervised multi-label image classification method as described in any one of claims 1 to 6.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the semi-supervised multi-label image classification method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the semi-supervised multi-label image classification method as described in any one of claims 1 to 6.

Images