Method, device and related product for multi-label classification of images

Abstract

The application provides a method and device for multi-label classification of images and related products, which can be applied to the field of artificial intelligence. The method comprises the following steps: first, obtaining an image to be classified; then inputting the image to be classified into a preset multi-label classification model to obtain a multi-label classification result of the image. The training sample set of the preset multi-label classification model is a first sample set composed of samples with a number of training labels less than the total number of actual labels. That is, part of the labels in the sample labels are used as training samples. Therefore, using part of the labels for model training can reduce the probability of inaccurate labels participating in training and improve the accuracy of sample labels in the training samples. In this way, training the multi-label classification model using the above training sample set can reduce the problem of overfitting of the model caused by label annotation bias and improve the classification accuracy of image multi-label classification.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus and related products for multi-label classification of images. Background Technology

[0002] In recent years, deep learning models have been increasingly widely used in the field of labeled image classification due to their advantages such as end-to-end processing, high accuracy, and high practicality. For example, deep neural network models are used in single-label image classification. However, in real-world scenarios, multi-label classification of images is often required. In this case, a label represents an attribute of the image content.

[0003] Currently, trained multi-label classification models can be used for image multi-label classification, such as a trained Xception neural network model. However, these trained multi-label classification models often suffer from overfitting due to label annotation bias. Therefore, using such models for image multi-label classification results in low classification accuracy. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus and related products for multi-label image classification, which aim to reduce the degree of overfitting problem in multi-label classification models during training and improve the classification accuracy of multi-label image classification.

[0005] In a first aspect, embodiments of this application provide a multi-label classification method for images, the method comprising:

[0006] Obtain the image to be classified;

[0007] The image to be classified is input into a preset multi-label classification model, and at least one label of the image to be classified is output to obtain the multi-label classification result of the image to be classified; the label is used to represent a content attribute of the image.

[0008] The training sample set of the preset multi-label classification model includes a first sample set; the first sample set is a set of samples with fewer training labels than the total number of actual labels.

[0009] Optionally, the preset multi-label classification model is trained in the following way:

[0010] Obtain the first sample set and the test set;

[0011] For the current training round, a multi-label classification model is trained based on the first sample set and the first loss function to obtain a preset multi-label classification model; the first loss function is related to the sample label ratio of each sample in the first sample set; the sample label ratio is the ratio of the number of trained labels in the sample to the total number of actual labels;

[0012] Based on the test set, a classification evaluation index for the preset multi-label classification model is determined; the classification evaluation index is used to represent the performance of the preset multi-label classification model.

[0013] In response to the classification evaluation index meeting the preset conditions, the training of the multi-label classification model is completed.

[0014] Optionally, the first loss function is obtained in the following way:

[0015] Determine the sample label ratio for each sample in the first sample set;

[0016] Based on the preset mapping relationship between sample label ratio and sample weight, the sample weight of each sample in the first sample set is determined.

[0017] The first loss function is obtained by weighting the second loss function of each sample in the first sample set using the determined sample weights of each sample.

[0018] Optionally, the first sample set includes a first sample, and training a multi-label classification model based on the first sample set and a first loss function includes:

[0019] Based on the first sample, construct a multi-label vector set for the first sample; the multi-label vector set for the first sample is a set of multi-dimensional vectors corresponding to each training label in the first sample;

[0020] The multi-label vector set of the first sample is input into the multi-label classification model, and the network output layer outputs the predicted probability of each training label in the first sample; the predicted probability of each training label is used to indicate the probability that the training label is a positive class.

[0021] The multi-label classification model is trained based on the predicted probability of each training label and the first loss function.

[0022] Optionally, the training sample set of the preset multi-label classification model further includes a second sample set; the second sample set is a set of samples in which the number of training labels is equal to the total number of actual labels.

[0023] The training method for the preset multi-label classification model also includes:

[0024] For the current training round, a multi-label classification model is trained using a third loss function based on the first sample set and the second sample set to obtain a preset multi-label classification model; the third loss function includes a first loss function and a fourth loss function.

[0025] Optionally, the third loss function further includes regularization weights, which are used to describe the degree of contribution of the loss of the first sample set to the final loss;

[0026] The third loss function is obtained in the following way:

[0027] The product of the regularization weights and the first loss function is added to the fourth loss function to obtain the third loss function.

[0028] Secondly, embodiments of this application also provide a multi-label image classification device, the device comprising:

[0029] The acquisition unit is used to acquire the image to be classified.

[0030] A classification unit is used to input the image to be classified into a preset multi-label classification model, output at least one label of the image to be classified, and obtain the multi-label classification result of the image to be classified; the label is used to represent a content attribute of the image;

[0031] The training sample set of the preset multi-label classification model includes a first sample set; the first sample set is a set of samples with fewer training labels than the total number of actual labels.

[0032] Optionally, the device further includes a training unit for training a multi-label classification model; the training unit includes:

[0033] The acquisition module is used to acquire the first sample set and the test set;

[0034] The training module is used for training in the current round, training a multi-label classification model based on the first sample set and the first loss function to obtain a preset multi-label classification model; the first loss function is related to the sample label ratio of each sample in the first sample set; the sample label ratio is the ratio of the number of trained labels in the sample to the total number of actual labels;

[0035] The determination module is used to determine the classification evaluation index of the preset multi-label classification model based on the test set; the classification evaluation index is used to represent the performance of the preset multi-label classification model.

[0036] The response module is used to complete the training of the multi-label classification model in response to the classification evaluation index meeting the preset conditions.

[0037] Thirdly, embodiments of this application also provide an electronic device, including: at least one processor and a memory communicatively connected to the at least one processor;

[0038] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any of the first aspects.

[0039] Fourthly, embodiments of this application also provide a computer-readable storage medium. The computer-readable storage medium stores computer instructions that are used to cause the computer to perform the method as described in any of the first aspects.

[0040] This application provides a method, apparatus, and related products for multi-label image classification. When performing the method, an image to be classified is first acquired. Then, the image to be classified is input into a preset multi-label classification model to obtain the multi-label classification result. The preset multi-label classification model uses a first sample set consisting of samples with fewer training labels than the total number of actual labels as the training sample set. That is, a subset of the sample labels is used as training samples. Therefore, during model training, selecting a subset of labels can reduce the probability of inaccurate labels participating in training, thereby improving the accuracy of the sample labels in the training samples. Thus, using the above-mentioned training samples to train the multi-label classification model can mitigate the degree of overfitting in the model and improve the classification accuracy of multi-label image classification. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0043] Figure 2 A flowchart of a multi-label image classification method provided in this application embodiment;

[0044] Figure 3 A flowchart illustrating a training method for a multi-label classification model provided in this application embodiment;

[0045] Figure 4 A flowchart illustrating a training method for another multi-label classification model provided in this application embodiment;

[0046] Figure 5 This application also provides a schematic diagram of a device structure for multi-label image classification. Detailed Implementation

[0047] As mentioned earlier, existing technologies often employ trained multi-label classification models for image multi-label classification. The inventors discovered that the training sample set for these models consists of fully labeled samples. That is, the multi-label classification model is trained using all labels from all samples in the sample set. However, on the one hand, collecting all labels from all samples in the sample set is difficult and requires significant resources and costs. Therefore, considering the collection cost, the size of the training sample set is reduced, resulting in lower fault tolerance and robustness of the trained model. On the other hand, fully labeled samples also cannot guarantee the accuracy of each label. For example, lavender and tulips, which are similar in color, may be mixed up. Ambiguous or mislabeled labels can introduce labeling bias into the samples, often leading to overfitting in the trained multi-label classification model.

[0048] Therefore, this application proposes a method for training multi-label classification models using a sample set composed of partially labeled samples. Specifically, a sample set consisting of samples with fewer training labels than the total number of actual labels is used as the training sample set for multi-label classification model training. By excluding uncertain or inaccurate labels from the samples and using the more accurate labels from the samples for training, the degree of overfitting in the model can be reduced, and the classification accuracy of image multi-label classification can be improved.

[0049] To enable those skilled in the art to better understand the technical solution of this application, the application scenarios of the technical solution of this application are first described in the accompanying drawings of the embodiments of this application.

[0050] See Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 1 As shown, the application scenario includes a terminal device 110, which can be any electronic device with processing capabilities, including but not limited to smartphones, tablets, laptops, desktop computers, and servers.

[0051] The terminal device 110 can, for example, process the input image 120. Specifically, it can recognize the input image and add multiple labels 130 to the image based on the recognition results. These multiple labels 130 can respectively indicate multiple attributes of the content in the image. Specifically, the terminal device 110 can, for example, determine the multiple labels added to the image based on a multi-label classification model. By adding these multiple labels, for example, it can detect objects in the image 120, determine which labels the image 120 contains, and perform multi-label classification on the image.

[0052] This application scenario also includes server 140. Terminal device 110 can communicate with server 140 via a network, which may include wired or wireless communication links. For example, server 140 can be used to train a multi-label classification model and, in response to a model acquisition request sent by terminal device 110, send the trained multi-label classification model 150 to terminal device 110, facilitating terminal device 110 to recognize the input image and determine the multiple labels to be added to image 120. Server 140 can also be used for multimodal feature extraction and fusion 180, and for knowledge graph construction 170.

[0053] For example, the server may be a server that provides various services, such as a backend management server that supports applications running on terminal device 110. For example, the server may be a cloud server, a server for a distributed system, or a server incorporating blockchain technology.

[0054] This application scenario also includes a database 160, which may maintain a massive number of images, including labeled images whose labels indicate an attribute of the image content. A server 140 may access the database 160 and randomly select a portion of images from the massive image database, using these selected images as training samples to train a multi-label classification model.

[0055] It should be noted that the training method for the multi-label classification model provided in this application can be executed by the server 140. Accordingly, the training device for the multi-label classification model provided in this application can be located in the server 140. The label prediction method provided in this application can be executed by the terminal device 110. Accordingly, the label prediction device provided in this application can be located in the terminal device 110.

[0056] It should be understood that Figure 1 The number and types of terminal devices, servers, and databases shown are merely illustrative. Depending on implementation needs, there can be any number and type of terminal devices, servers, and databases.

[0057] Multi-label classification models can be applied in various fields, such as content recognition and automatic classification of mobile phone album images, or as an upstream task to provide data and feature support for downstream artificial intelligence tasks such as knowledge graph construction, multimodal feature extraction and fusion.

[0058] Next, with reference to the accompanying drawings of the embodiments of this application, the technical solutions of the embodiments of this application will be clearly and completely described. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] See Figure 2 This is a flowchart of a multi-label image classification method provided in an embodiment of this application. The method includes at least the following steps:

[0060] S201: Obtain the image to be classified.

[0061] Terminal devices can acquire images to be classified in various ways, such as by acquiring images through image acquisition devices or by retrieving images from a database. For example, landscape photos can be acquired through smartphones or cameras.

[0062] S202: Input the image to be classified into the preset multi-label classification model, output at least one label of the image to be classified, and obtain the multi-label classification result of the image to be classified.

[0063] The terminal device inputs the acquired image to be classified into a preset multi-label classification model. By identifying the attributes in the input image and based on the pre-determined correspondence between labels and attributes, it outputs at least one label for the image to be classified. These output labels are the multi-label classification results of the image to be classified.

[0064] Tags are used to describe a content attribute of an image. The correspondence between tags and content attributes is predetermined. For example, the correspondence between tags and content attributes can be constructed by adding tags. Specific methods... Figure 1 As already described in detail, I will not go into further detail here.

[0065] In this embodiment, the pre-defined multi-label classification model can be obtained through training. The training samples include a first sample set. The first sample set is a collection of samples where the number of training labels is less than the total number of actual labels. For example, consider a sample set containing 100 image samples, each with 10 labels. In this set, 20 samples have 3 labels that are either unlabeled or manually masked. That is, if these 20 samples are used as training samples, the unlabeled or masked labels will not be identified or trained; the number of labels participating in training is less than the total number of actual labels. These 20 samples constitute the first sample set. In practical applications, selecting a subset of labels can reduce the probability of ambiguous or mislabeled labels participating in training. Therefore, using the first sample set as training samples can reduce labeling bias, improve sample accuracy, and thus mitigate the overfitting problem in the trained multi-label classification model.

[0066] In this embodiment, the selected multi-label classification model can be the Xception neural network model. The Xception neural network is an improved version of the Inception network, which completely separates channel correlation from spatial correlation by adjusting the ratio of the number of 3x3 convolutional branches to the number of 1x1 convolutional channels. Compared to the Inception network, it reduces the model parameter size and improves the performance of model training and inference. In this embodiment, other deep convolutional neural networks can also be used as the multi-label classification model.

[0067] This application provides a method for multi-label image classification. First, an image to be classified is acquired. Then, the image is input into a pre-defined multi-label classification model to obtain the multi-label classification result. The pre-defined multi-label classification model uses a first sample set consisting of samples with fewer training labels than the total number of actual labels as training samples. Therefore, during model training, selecting a subset of labels can reduce the probability of uncertain or inaccurate labels participating in training, thereby improving the accuracy of the sample labels in the training samples. Thus, using the aforementioned training samples to train the multi-label classification model can mitigate the overfitting problem, thereby improving the classification accuracy of multi-label image classification.

[0068] about Figure 2 There are various ways to train a multi-label classification model. To better illustrate the training of a multi-label classification model, the following description is provided with reference to the accompanying drawings in the embodiments of this application.

[0069] See Figure 3 This is a flowchart illustrating a training method for a multi-label classification model provided in an embodiment of this application. Applied to the Xception neural network model, the method includes at least the following steps:

[0070] S301: Obtain the first sample set and the test set.

[0071] In this embodiment, the first sample set is a collection of samples where the number of training labels is less than the total number of actual labels. The test set is used to test the performance of the trained multi-label classification model.

[0072] In this embodiment, the labels of the first sample set are processed into text format, converting the text format label of each sample into 0 and 1 numerical features, ultimately forming a multi-label vector set. For example, if the total number of image labels in the first sample set is 1600, then the label of each sample can be represented as a 1600-dimensional sparse multi-hot vector, where 1 represents that the label is labeled and 0 represents that it is not labeled.

[0073] S302: For the current training round, train a multi-label classification model based on the first sample set and the first loss function to obtain the trained multi-label classification model.

[0074] In this embodiment, the multi-label classification model is an Xception neural network model. Specifically, the first sample set and the corresponding label vectors are input into the Xception neural network for convolution processing. After activation by an activation function, the network output layer outputs the predicted probability of each training label in the first sample set. The predicted probability indicates the probability that a training label is a positive class. In this embodiment, the activation function can be the Sigmoid activation function. The current model loss is calculated using the predicted probability results and the true label vectors. The loss is then minimized using a stochastic gradient descent algorithm to update the model parameter values.

[0075] In this embodiment, a first loss function, Partial Label Loss, is introduced. This loss function is related to the ratio of the number of trained labels to the total number of actual labels in the first sample set.

[0076] In this embodiment of the application, the first loss function can be obtained by: determining the sample label ratio of each sample in the first sample set; determining the sample weight of each sample in the first sample set according to the mapping relationship between the sample label ratio and the sample weight; and using the sample weight to perform a weighted average processing on the second loss function of each sample in the first sample set to obtain the first loss function.

[0077] Example illustration: The preset mapping relationship between sample weights and sample label ratios is as follows:

[0078]

[0079] Where py is the partial label ratio of each sample in the first sample set, determined by calculating the ratio of the number of training labels to the total number of actual labels. α, β, and γ are the model mapping parameters, which can be set by those skilled in the art as needed. g(py) is the sample weight of that sample.

[0080] The first loss function, Partial Label Loss, is:

[0081]

[0082] Where C is the complete set of sample labels in the first sample set; This indicates that the sample has been labeled with the current label; This sample was not labeled with the current tag.

[0083] In this embodiment of the application, the first sample set includes a first sample, and a multi-label vector set of the first sample is constructed based on the labels of the first sample. The multi-label vector set is a collection of multi-dimensional vectors corresponding to each training label in the first sample.

[0084] The multi-label vector set is input into a multi-label classification model, and the network output layer outputs the predicted probability of each training label in the first sample. The multi-label classification model can be trained using the predicted probability of each training label and a first loss function.

[0085] S303: Based on the test set, determine the classification evaluation metrics for the trained multi-label classification model.

[0086] In this embodiment, the test set is input into the trained multi-label classification model, and the training result is judged. Specifically, the performance improvement of the multi-label classification model can be determined by classification evaluation metrics. In this embodiment, the classification evaluation metrics can be the AUC value and F1 value of the classification labels.

[0087] S304: In response to the classification metric meeting the preset conditions, complete the training of the multi-label classification model.

[0088] In this embodiment, the classification metric is a classification evaluation metric. Specifically, it is determined whether the classification evaluation metric has reached a preset threshold. If so, the training of the multi-label classification model is completed; otherwise, the model mapping parameters need to be adjusted, and the classification model continues to be trained until the preset threshold is met.

[0089] In this embodiment, by introducing the first loss function, Partial Label Loss, the impact of the sample set composed of partial label samples on the entire training can be determined, the degree of overfitting problem in the model can be reduced, and the classification accuracy of the multi-label classification model after training can be further improved.

[0090] See Figure 4 This is a flowchart illustrating a training method for another multi-label classification model provided in this application embodiment. Applied to the Xception neural network model, the method includes at least the following steps:

[0091] S401: Obtain the training sample set and the test set, wherein the training sample set includes the first sample set and the second sample set.

[0092] In this embodiment, the first sample set is a set of samples where the number of training labels is less than the total number of actual labels. The second sample set is a set of samples where the number of training labels is equal to the total number of actual labels. For example, if the training sample set has 100 samples, and 20 samples have fewer training labels than the total number of actual labels, then the first sample set is the set of these 20 samples. The second sample set is the remaining 80 samples. In this embodiment, the training samples refer to both the first and second sample sets.

[0093] S402: For the current training round, train a multi-label classification model based on the training sample set and the third loss function to obtain the trained multi-label classification model.

[0094] In the current training round, the obtained training sample set is input into the multi-label classification model. Convolutional processing is performed, and the network output layer, after activation by an activation function, outputs the predicted probability of each training label for each sample in the target sample set. The current model loss is calculated using the predicted probability results and the true label vector. The loss is then minimized using the stochastic gradient descent algorithm to update the model parameter values.

[0095] In this embodiment, a third loss function is introduced. This third loss function includes both a first loss function and a fourth loss function.

[0096] In this embodiment, the third loss function is the sum of the product of the regularization weights and the first loss function, and the fourth loss function. The regularization weights represent the contribution of the loss of the first sample set to the final loss. The first loss function and... Figure 3 The first loss function is the same as described above, and will not be discussed further here.

[0097] Example illustration: The third loss function can be expressed as:

[0098] l = l multi-sigmoid +ωl partial-label

[0099] The fourth loss function is the Multi-Sigmoid loss function, where ω is a regularization weight that controls the influence of the first loss function on the third loss function. Those skilled in the art can adjust this weight as needed. The Multi-Sigmoid loss function breaks down the multi-label classification task into multiple binary classification tasks, with each binary classification task using a binary cross-entropy loss to calculate its loss.

[0100] By introducing a first loss function into the existing loss function, and using this first loss function to constrain the existing loss function, a regularization term can be applied to some extent. This reduces the labeling bias that might be introduced by training with all labels, mitigating the degree of overfitting and thus improving the accuracy of multi-label image classification.

[0101] S403: Based on the test set, determine the classification evaluation metrics for the trained multi-label classification model.

[0102] S404: In response to the classification index meeting the preset conditions, the training of the multi-label classification model is completed.

[0103] In the above steps, S403 and S404 are related to Figure 3 The implementation methods of S303 and S304 are the same, so they will not be discussed here.

[0104] To illustrate more vividly Figure 4 The training of the multi-label classification model is illustrated below using a 480P video cover image as an input sample:

[0105] The target sample set consisted of 10 million images, and the test set consisted of 1 million images. Partially labeled samples (with fewer than 3 labels) accounted for approximately 70% of the total samples. Specifically, samples with one label accounted for 27.91%, and samples with two labels accounted for 40.75%. The total number of image label categories was 1600, including categories such as historical drama, urban drama, and story drama. Each image's label was represented as a 1600-dimensional sparse multi-hot vector. Labeled labels were assigned a dimension of 1, and unlabeled labels were assigned a dimension of 0.

[0106] The target sample set and corresponding labels are input into the Xception neural network model for supervised learning training. The network output layer uses the Multi-Sigmoid function as the activation function. The model output is a 1600-dimensional sparse vector, with values ​​ranging from 0 to 1 for each dimension. Values ​​closer to 1 indicate a higher probability that the model predicts the input image belongs to that label.

[0107] The loss function used in this model is:

[0108] l = l multi-sigmoid+ωl partial-label

[0109] The AUC and F1 scores of the labels are used as the model classification evaluation metrics. After small-scale pre-training and mapping parameter search, an ideal set of parameters (α = 0.1, β = 1, γ = -1, ω = 0.1) is obtained. After training the above model for 100,000 steps, the AUC and F1 scores of the labels tend to stabilize and can continue to improve slightly. After 200,000 steps, compared with the existing model, the new model improves the AUC by 0.30% and the F1 by 1.53%. See Table 1 for the improvement results of AUC and F1 scores of some label models provided in the embodiments of this application. In the embodiments of this application, the existing model is a model obtained by training the trained Xception neural network model with a full-label sample training set.

[0110]

[0111] Table 1

[0112] This application uses a large number of partially labeled image samples for training, and adds PartialLabel Loss to the model and improves the regularization of Multi-Sigmoid Loss to enhance the label AUC and F1 values, thereby further improving the accuracy of multi-label classification.

[0113] This application also provides a schematic diagram of the structure of a multi-label image classification device 500, see [link / reference]. Figure 5 .Depend on Figure 5 It can be seen that the device includes:

[0114] The acquisition unit 501 is used to acquire the image to be classified.

[0115] The classification unit 502 is used to input the image to be classified into a preset multi-label classification model, output at least one label of the image to be classified, and obtain the multi-label classification result of the image to be classified; a label is used to represent a content attribute of the image.

[0116] The training sample set of the preset multi-label classification model includes a first sample set; the first sample set is a set of samples with fewer training labels than the total number of actual labels.

[0117] Optionally, the device 500 further includes a training unit for training a multi-label classification model. The training unit includes:

[0118] The acquisition module is used to acquire the first sample set and the test set.

[0119] The training module is used for training in the current round. It trains a multi-label classification model based on the first sample set and the first loss function to obtain the trained multi-label classification model. The first loss function is related to the ratio of the sample labels of each sample in the first sample set. The sample label ratio is the ratio of the number of trained labels in the sample to the total number of actual labels.

[0120] A determination module is used to determine the classification evaluation index of the trained multi-label classification model based on the test set, wherein the classification evaluation index is used to represent the accuracy of the multi-label classification.

[0121] The response module is used to complete the training of the multi-label classification model in response to the fact that the classification evaluation index meets the preset conditions.

[0122] Optionally, the first loss function is obtained in the following way:

[0123] Determine the sample label ratio for each sample in the first sample set;

[0124] Based on the preset mapping relationship between sample label ratio and sample weight, the sample weight of each sample in the first sample set is determined.

[0125] The first loss function is obtained by weighting the second loss function of each sample in the first sample set using the determined sample weights.

[0126] Optionally, the first sample set includes a first sample, and training the multi-label classification model based on the first sample set and a first loss function includes:

[0127] Based on the first sample, a multi-label vector set of the first sample is constructed; the multi-label vector set is a collection of multi-dimensional vectors corresponding to each training label in the first sample;

[0128] The multi-label vector set is input into the multi-label classification model, and the network output layer outputs the predicted probability of each training label in the first sample; the predicted probability is used to indicate the probability that the training label is a positive class.

[0129] The multi-label classification model is trained based on the predicted probability of each training label and the first loss function.

[0130] Optionally, the training sample set of the preset multi-label classification model further includes a second sample set; the second sample set is a set of samples in which the number of training labels is equal to the total number of actual labels.

[0131] The training method for the preset multi-label classification model also includes:

[0132] For the current training round, based on the first sample set and the second sample set, the multi-label classification model is trained using a third loss function to obtain the trained multi-label classification model; the third loss function includes the first loss function and the fourth loss function.

[0133] Optionally, the third loss function further includes regularization weights, which are used to describe the degree of contribution of the loss of the first sample set to the final loss;

[0134] The third loss function is obtained by adding the product of the fourth loss function, the regularization weight, and the first loss function.

[0135] This application provides an image multi-label classification device. The acquisition unit 501 acquires an image to be classified. The classification unit 502 inputs the image to be classified into a preset multi-label classification model to obtain the multi-label classification result of the image. The preset multi-label classification model uses a first sample set consisting of samples with fewer training labels than the total number of actual labels as training samples. That is, it uses a subset of the sample labels as training samples. Therefore, when training the model, selecting a subset of labels can reduce the probability of uncertain or inaccurate labels participating in training, thereby improving the accuracy of the sample labels in the training samples. Thus, using the above-mentioned training samples to train the multi-label classification model can reduce the degree of overfitting, thereby improving the classification accuracy of image multi-label classification.

[0136] In addition, embodiments of this application also provide an electronic device and a computer storage medium for implementing the solutions provided in embodiments of this application.

[0137] An electronic device includes: at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a multi-label classification method for images.

[0138] The computer storage medium stores code, and when the code is run, the device running the code implements the multi-label classification method for images described in any embodiment of this application.

[0139] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0140] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0141] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0142] The above description is merely an exemplary implementation of this application and is not intended to limit the scope of protection of this application.

Claims

1. A multi-label classification method for images, characterized in that, The method includes: Obtain the image to be classified; The image to be classified is input into a preset multi-label classification model, and at least one label of the image to be classified is output to obtain the multi-label classification result of the image to be classified; the label is used to represent a content attribute of the image. The training sample set of the preset multi-label classification model includes a first sample set and a second sample set; the first sample set is a set of samples in which the number of training labels is less than the total number of actual labels, and the second sample set is a set of samples in which the number of training labels is equal to the total number of actual labels; the training method of the multi-label classification model includes: training the multi-label classification model based on the first sample set and the second sample set using a third loss function. The third loss function is the product of the regularization weights and the first loss function, plus the fourth loss function. The regularization weights are used to describe the contribution of the loss of the first sample set to the final loss. The first loss function is obtained by: determining the sample label ratio of each sample in the first sample set; determining the sample weight of each sample in the first sample set according to a preset mapping relationship between the sample label ratio and the sample weight; and using the determined sample weights of each sample to perform a weighted average of the second loss function of each sample in the first sample set to obtain the first loss function. The fourth loss function is a Multi-Sigmoid loss function.

2. The method according to claim 1, characterized in that, The training method for the pre-defined multi-label classification model also includes: Get the test set; Based on the test set, a classification evaluation index for the preset multi-label classification model is determined; the classification evaluation index is used to represent the performance of the preset multi-label classification model. In response to the classification evaluation index meeting the preset conditions, the training of the multi-label classification model is completed.

3. The method according to claim 1, characterized in that, The first sample set includes a first sample, and the multi-label classification model obtained by training based on the first sample set and the second sample set using a third loss function includes: Based on the first sample, construct a multi-label vector set for the first sample; the multi-label vector set for the first sample is a set of multi-dimensional vectors corresponding to each training label in the first sample; The multi-label vector set of the first sample is input into the multi-label classification model, and the network output layer outputs the predicted probability of each training label in the first sample; the predicted probability of each training label is used to indicate the probability that the training label is a positive class. The multi-label classification model is trained based on the predicted probability of each training label, combined with the second sample set, and the third loss function.

4. An apparatus for multi-label classification of images, characterized in that, The device includes: The acquisition unit is used to acquire the image to be classified. A classification unit is used to input the image to be classified into a preset multi-label classification model, output at least one label of the image to be classified, and obtain the multi-label classification result of the image to be classified; the label is used to represent a content attribute of the image; The training sample set of the preset multi-label classification model includes a first sample set; the first sample set is a set of samples in which the number of training labels is less than the total number of actual labels; the training sample set of the preset multi-label classification model includes a first sample set and a second sample set; the first sample set is a set of samples in which the number of training labels is less than the total number of actual labels, and the second sample set is a set of samples in which the number of training labels is equal to the total number of actual labels; the training method of the multi-label classification model includes: training the multi-label classification model using a third loss function based on the first sample set and the second sample set; The third loss function is the product of the regularization weights and the first loss function, plus the fourth loss function. The regularization weights are used to describe the contribution of the loss of the first sample set to the final loss. The first loss function is obtained by: determining the sample label ratio of each sample in the first sample set; determining the sample weight of each sample in the first sample set according to a preset mapping relationship between the sample label ratio and the sample weight; and using the determined sample weights of each sample to perform a weighted average of the second loss function of each sample in the first sample set to obtain the first loss function. The fourth loss function is a Multi-Sigmoid loss function.

5. An electronic device, comprising: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method as described in any one of claims 1-3.

Images