Method and device for adding classification label through classification model, equipment and medium
By extracting the features of the semi-label sample and updating the classifier, adding missing tags to the semi-label sample, the problem of lack of training data of multi-classification models is solved, and classification accuracy and generalization are improved.
Patent Information
- Application Number
- CN202310799216.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-30
- Publication Date
- 2024-12-31
AI Technical Summary
In the training of existing multi-class classification models, some data of category labels are often discarded, resulting in a lack of training data, low classification accuracy and poor generalization of the model.
The features of the semi-label sample are extracted through the feature extraction network, and the classification loss is calculated using part classifiers, the classifier is updated, and the missing classification tags are added to the semi-label sample, thereby pre-training and retraining of the model.
The use of semi-label data for pre-training and retraining significantly improves the classification accuracy and generalization of multi-category classification models and expands the scale of training data.
Smart Images
Figure CN119226840A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of machine learning technology, and more particularly to a method, apparatus, device and medium for adding classification labels through a classification model. Background Art
[0002] With the development of digital technology, massive amounts of data, such as multi-modal data, have been generated. Faced with such a huge amount of data, it is difficult to classify these data manually. Therefore, data classification through machine learning is a common task in many applications. And these data can often belong to multiple categories, that is, they can have multiple categories of classification labels. The existing training schemes for multi-classification classification models usually rely on data with full category classification labels for training. However, in reality, data marked with complete category classification labels are often difficult to obtain, and many data only have labels for some categories. Therefore, in the training of existing multi-classification classification models, data with only partial category labels are often discarded, and only data with full category classification labels are used for training. This results in a lack of training data for the classification model, resulting in poor training results, especially the low classification accuracy and poor generalization of the model. Summary of the invention
[0003] The embodiments of the present disclosure describe a method, apparatus, device and medium for adding classification labels through a classification model.
[0004] According to a first aspect, there is provided a method for adding classification labels through a classification model, wherein the classification model includes a predetermined number of classifiers corresponding to a predetermined number of categories, the method comprising:
[0005] Acquire a first batch of samples, wherein the first batch of samples includes a plurality of semi-labeled samples, and the semi-labeled samples have classification labels of some categories among the predetermined number of categories;
[0006] Extracting a plurality of first sample features corresponding to the plurality of half-labeled samples respectively through a feature extraction network, inputting each first sample feature into a portion of the predetermined number of classifiers, and obtaining a plurality of classification results corresponding to each half-labeled sample, wherein the portion of the classifiers respectively corresponds to a portion of the categories with classification labels of each half-labeled sample; determining a classification loss according to the plurality of classification results corresponding to each half-labeled sample and the classification labels of the portion of the categories of each half-labeled sample; and updating the predetermined number of classifiers according to the classification loss;
[0007] Based on the updated predetermined number of classifiers, missing classification labels are added to the semi-labeled samples included in the second batch of samples.
[0008] According to a third aspect, there is provided a device for adding classification labels through a classification model, wherein the classification model includes a predetermined number of classifiers corresponding to a predetermined number of categories, and the device includes:
[0009] An acquisition unit is configured to acquire a first batch of samples, wherein the first batch of samples includes a plurality of semi-labeled samples, and the semi-labeled samples have classification labels of some categories among a predetermined number of categories;
[0010] The model updating unit is configured to respectively extract a plurality of first sample features corresponding to the plurality of half-labeled samples through a feature extraction network, respectively input each first sample feature into a portion of the predetermined number of classifiers to obtain a plurality of classification results corresponding to each half-labeled sample, wherein the portion of the classifiers respectively corresponds to a portion of the categories with classification labels of each half-labeled sample; determine a classification loss according to the plurality of classification results corresponding to each half-labeled sample and the classification labels of the portion of the categories of each half-labeled sample; and update the predetermined number of classifiers according to the classification loss;
[0011] The label adding unit is configured to add missing classification labels to the semi-labeled samples included in the second batch of samples based on the updated predetermined number of classifiers.
[0012] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect.
[0013] According to a fourth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect is implemented.
[0014] According to an embodiment of the present disclosure, a method, device, equipment and medium for adding classification labels through a classification model are provided. A first batch of samples including multiple semi-labeled samples can be obtained, and multiple first sample features corresponding to the multiple semi-labeled samples can be extracted through a feature extraction network, and each first sample feature is input into a part of the classifiers in a predetermined number of classifiers to obtain multiple classification results corresponding to each semi-labeled sample. And, according to the multiple classification results corresponding to each semi-labeled sample and the classification labels of some categories of each semi-labeled sample, the classification loss is determined; according to the classification loss, the predetermined number of classifiers are updated. Then, based on the updated predetermined number of classifiers, the missing classification labels are added to the semi-labeled samples included in the second batch of samples. Using any one of the above methods, devices, storage media and electronic devices, the missing classification labels can be added to the semi-labeled samples, and then the semi-labeled samples after adding the labels can be used to train the classification model, which greatly improves the training effect of the model, especially improves the classification accuracy and generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 A schematic diagram of a process of adding classification labels through a classification model according to an embodiment of the present disclosure is shown;
[0016] Figure 2 A schematic diagram of multimodal feature fusion according to an embodiment of the present disclosure is shown;
[0017] Figure 3 A schematic diagram of pre-training according to an embodiment of the present disclosure is shown;
[0018] Figure 4 A schematic diagram of semi-label sample marking according to an embodiment of the present disclosure is shown;
[0019] Figure 5 A schematic diagram of retraining a model using labeled half-label samples according to an embodiment of the present disclosure is shown;
[0020] Figure 6 A schematic block diagram of a device for adding classification labels through a classification model according to an embodiment of the present disclosure is shown;
[0021] Figure 7 A schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present disclosure is shown;
[0022] Figure 8 A schematic diagram of the structure of a storage medium suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0023] The technical solution provided in this specification is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the inventions. It should also be noted that, for ease of description, only the parts related to the relevant inventions are shown in the accompanying drawings. It should be noted that, in the absence of conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.
[0024] In the description of the implementations of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one / kind of implementation" or "the implementation" should be understood as "at least one / kind of implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below.
[0025] As mentioned above, data classification through machine learning is a common task in many applications. In reality, data used for classification often has classification labels of multiple categories. For example, a piece of media data can have classification labels of multiple categories, such as classification labels of scene categories (for example, specifically "travel"), and classification labels of style categories (for example, specifically "fashion"), etc. The existing training schemes for multi-classification classification models usually rely on data with full category classification labels for training. However, data marked with complete category classification labels that can be used for classification model training is often difficult to obtain, and many data only have labels of some categories. For example, some media data only have classification labels of scene categories, and other media data only have classification labels of style categories. The existing training schemes for multi-classification classification models often discard data with only partial category labels and only use data with full category classification labels for training. However, due to the scarcity of data with full category classification labels, the training data of the classification model is insufficient, and the training effect is poor, especially the classification accuracy and generalization of the trained model are poor.
[0026] In order to solve the above technical problems, the embodiments of the present disclosure provide a method for adding classification labels through a classification model. In some embodiments, the classification model can be pre-trained by using half-labeled data (Half Labeled Data, HLD) with classification labels of some categories and full-labeled data (Full Labeled Data, FLD) with classification labels of all categories. In the calculation of the classification loss in pre-training, the classification loss can be calculated according to the partial classification labels of the half-labeled data and the full classification labels of the full-labeled data. And based on the classification loss, all classifiers of the classification model are updated. Then, the half-labeled data is classified by the updated classification model, and the half-labeled data is labeled with the classification results to add the classification labels missing from the half-labeled data. In other embodiments, the classification model can also be retrained (or re-trained) using the half-labeled data after adding the missing classification labels. The advantage of this method is that the classification model can be pre-trained using the half-labeled data. And the pre-trained classification model is used to label the half-labeled data to supplement its missing classification labels, which essentially converts the half-labeled data into full-labeled data. After that, the classification model can be retrained using the semi-labeled data that supplements the missing classification labels. Therefore, through this method, the semi-labeled data can be automatically labeled during training, the semi-labeled data can be converted into full-labeled data, and the classification model can be further trained using the converted full-labeled data. That is, the multi-category classification model can be trained using semi-labeled data. In this way, the training data of the multi-category classification model is greatly expanded, and the useful information in the existing semi-labeled data and the full semi-labeled data is used simultaneously in the training, which significantly improves the classification accuracy and generalization of the multi-category classification model.
[0027] The detailed process of this method is further described below.
[0028] Figure 1 A flow chart of a method for adding classification labels through a classification model according to an embodiment of the present disclosure is shown. The classification model includes a predetermined number of classifiers corresponding to a predetermined number of categories, such as Figure 1 As shown, the method comprises at least the following steps:
[0029] Step S101, obtaining a first batch of samples, wherein the first batch of samples includes a plurality of semi-labeled samples, and the semi-labeled samples have classification labels of some categories among the predetermined number of categories;
[0030] Step S103, extracting a plurality of first sample features corresponding to the plurality of half-labeled samples respectively through a feature extraction network, inputting each first sample feature into a portion of the predetermined number of classifiers, and obtaining a plurality of classification results corresponding to each half-labeled sample, wherein the portion of the classifiers respectively corresponds to a portion of the categories with classification labels of each half-labeled sample; determining a classification loss according to the plurality of classification results corresponding to each half-labeled sample and the classification labels of the portion of the categories of each half-labeled sample; and updating the predetermined number of classifiers according to the classification loss;
[0031] Step S105 , based on the updated predetermined number of classifiers, adding missing classification labels to the semi-labeled samples included in the second batch of samples.
[0032] First, in step S101, a first batch of samples can be obtained. The first batch of samples may include a plurality of semi-label samples, and the semi-label samples may have classification labels of some categories in the predetermined number of categories. In different embodiments, the samples included in the first batch of samples may be data for different applications or services, and this specification does not limit this. In one embodiment, for example, it may be multimodal data. In different specific embodiments, the multimodal data may include data of different specific modalities. In a specific embodiment, the samples included in the first batch of samples may be, for example, media data samples, and the media data samples may include multiple modal data, and the multiple modal data include two or more of visual modal data, textual modal data, motion modal data, and audio modal data.
[0033] In different embodiments, the predetermined number of categories may be different specific categories. For example, in the above-mentioned example of media data, the predetermined number of categories may include, for example, scene categories, character categories, and style categories. In different embodiments, the classification labels of some categories possessed by each semi-labeled sample may also be different. For example, in the above-mentioned example of media data, one semi-labeled sample may have, for example, classification labels of scene categories and character categories, and another semi-labeled sample may have, for example, classification labels of character categories and style categories.
[0034] In one embodiment, the first batch of samples may further include a plurality of fully labeled samples, wherein the fully labeled samples have classification labels of a predetermined number of categories. By including fully labeled samples in the first batch of samples, effective information in fully labeled samples and semi-labeled samples can be simultaneously utilized during the model pre-training process in subsequent steps to improve the pre-training effect of the classification model, that is, to improve the classification accuracy and generalization of the pre-trained model.
[0035] Then, in step S103, the classification model can be pre-trained. Specifically, the multiple first sample features corresponding to the multiple semi-labeled samples can be respectively extracted through the feature extraction network, and each first sample feature can be respectively input into a part of the classifiers in the predetermined number of classifiers to obtain multiple classification results corresponding to each semi-labeled sample, and the part of the classifiers respectively correspond to the part of the categories with classification labels of each semi-labeled sample; the classification loss is determined according to the multiple classification results corresponding to each semi-labeled sample and the classification labels of the part of the categories of each semi-labeled sample; and the predetermined number of classifiers are updated according to the classification loss.
[0036] In different embodiments, the specific manner of extracting the first sample feature may be different. As mentioned above, in different embodiments, the samples included in the first batch of samples may be data for different applications or services, for example, multimodal data. Therefore, in one embodiment, the semi-labeled sample may include multiple modal data. In addition, the feature extraction network may include multiple sub-feature extraction networks. Furthermore, for any one of the multiple semi-labeled samples, multiple sub-modal features corresponding to the multiple modal data included in the any one semi-labeled sample may be extracted by multiple sub-feature extraction networks; the multiple sub-modal features are fused to obtain the first sample feature corresponding to the any one semi-labeled sample. In different specific embodiments, the specific network types of different sub-feature extraction networks may also be different according to the types of different modal data extracted, and this specification does not limit this. In different specific embodiments, the specific manner of fusing the multiple sub-modal features may be different. In a specific embodiment, the multiple sub-modal features may be fused by summing or cascading operations. In the above manner, the multimodal fusion features (for example, the first sample features) of the multimodal data may be extracted and used for the classification of the multimodal data.
[0037] Figure 2 A schematic diagram of a method for adding classification labels through a classification model according to an embodiment of the present disclosure is shown. Figure 2 In the illustrated embodiment, for example, a semi-labeled sample (including a fully labeled sample) specifically a media data sample may include visual modal data, visual modal data, and visual modal data. Thus, the feature extraction network may include, for example, a plurality of feature extraction subnetworks, such as feature extraction subnetwork 1, feature extraction subnetwork 2, and feature extraction subnetwork 3. The feature extraction subnetwork 1, feature extraction subnetwork 2, and feature extraction subnetwork 3 may be used to extract visual modal data, visual modal data, and a plurality of submodal features corresponding to the visual modal data, respectively, and the plurality of submodal features may be fused into a multimodal fusion feature of the media data sample.
[0038] As described above, in different embodiments, the first batch of samples may also include multiple full-label samples. Furthermore, in one embodiment, multiple second sample features corresponding to the multiple full-label samples may be extracted respectively through a feature extraction network, and each second sample feature may be input into the predetermined number of classifiers respectively to obtain the predetermined number of classification results corresponding to each full-label sample. Then, the classification loss is determined based on the multiple classification results corresponding to each semi-label sample, the classification labels of some categories possessed by each semi-label sample, and the predetermined number of classification results corresponding to each full-label sample, and the predetermined number of classification labels possessed by each full-label sample. In the above manner, during the model training (pre-training) process, the effective information in the full-label samples and the semi-label samples may be utilized simultaneously to improve the classification accuracy and generalization of the pre-trained model.
[0039] In different embodiments, the specific structure of the classifier may be different. In a specific embodiment, the classifier includes a plurality of fully-connected layers. In different embodiments, the classification loss in pre-training may be determined by different specific loss functions. In one embodiment, the classification loss may be determined in pre-training by a cross entropy loss function. In a specific embodiment, the classification loss may be determined in pre-training by the following formula:
[0040]
[0041] in,
[0042]
[0043]
[0044] Among them, L pre is the classification loss in pre-training, L FLD The classification loss part determined based on the full label samples, L HLD The classification loss part determined based on the semi-labeled samples, N represents the number of all samples in the first batch of samples, N F Indicates the number of fully labeled samples in the first batch of samples, N H represents the number of semi-labeled samples in the first batch of samples, i F is the ordinal number of the fully labeled sample, i H is the ordinal number of the semi-labeled sample, are the sample features of full-label samples and half-label samples respectively, and are the classification results of the fully labeled samples and the semi-labeled samples, respectively. θ(*) represents the parameters of the network. C represents the number of subcategories of any classifier, and j and k represent the subcategory ordinal numbers of the classifier. T represents the transposition symbol, Tc represents the number of classifiers, t represents the ordinal number of the classifier, label_mask (t) Indicates the category mask, label_mask (t) =1 means that the half-labeled sample before training has the classification label corresponding to the classifier with ordinal number t, label_mask (t) =0 means that before training, the half-label sample does not have the classification label corresponding to the classifier with ordinal number t.
[0045] In one embodiment, the feature extraction network for extracting sample features can be included in the classification model. Therefore, in one embodiment, the predetermined number of classifiers and the feature extraction network can also be updated according to the classification loss. In the above manner, the feature extraction network can be updated during model training, the feature extraction capability of the feature extraction network can be improved, and the classification accuracy of the classification model can be further improved.
[0046] Figure 3 FIG. 2 shows a schematic diagram of pre-training according to an embodiment of the present disclosure. Figure 3 As shown, for example, based on the semi-labeled samples in the first batch of samples, for example, including sample A (having classification labels a1 and a3 corresponding to some classifiers of the classification model, such as classifier 1 and classifier 3, respectively). Sample feature 1 corresponding to sample A is extracted, and then sample feature 1 is input into classifier 1 and classifier 3 respectively to obtain corresponding classification results 1 and classification results 2. The classification loss part L determined based on the semi-labeled samples is calculated based on classification results 1 and classification results 2, and classification labels a1 and a3. FLD . In addition, based on the full-label samples in the first batch of samples, for example, including sample AB (having classification labels b1, b2, and b3 corresponding to all classifiers of the classification model, such as classifier 1, classifier 2, and classifier 3, respectively). Extract the sample feature 2 corresponding to sample B, and then input the sample feature 2 into classifier 1, classifier 2, and classifier 3, respectively, to obtain the corresponding classification results 3, classification results 4, and classification results 5. According to classification results 3, classification results 4, and classification results 5, and classification labels b1, b2, and b3, calculate the classification loss part L determined based on the full-label samples FLD Then, for example, according to L FLD and L HLD , determine the classification loss L of the first batch of samples pre , and according to the classification loss L pre , update all classifiers of the classification model. Specifically, in one example. In one embodiment, L FLD and L HLD After applying different weights, according to the weighted L FLD and L HLD, determine the classification loss L of the first batch of samples pre Furthermore, L can be adjusted according to the actual training results. FLD and L HLD Apply different weights to further improve the training effect of pre-training, such as improving the accuracy of the pre-trained classification model.
[0047] Thereafter, in step S105, the missing classification labels can be added to the semi-labeled samples included in the second batch of samples based on the updated predetermined number of classifiers. Specifically, in one embodiment, the third sample features corresponding to the semi-labeled samples included in the second batch of samples can be respectively extracted through the feature extraction network, and the third sample features can be respectively input into the predetermined number of classifiers to obtain multiple classification results corresponding to the semi-labeled samples included in the second batch of samples; and the missing classification labels can be added to the semi-labeled samples included in the second batch of samples according to the multiple classification results corresponding to the semi-labeled samples included in the second batch of samples.
[0048] In different embodiments, the semi-labeled samples included in the second batch of samples may include the semi-labeled samples included in the first batch of samples, or the semi-labeled samples other than the first batch of samples. In the above manner, missing classification labels can be added to a large number of semi-labeled samples for the semi-labeled samples included in the first batch of samples and the semi-labeled samples other than the first batch of samples, and used in subsequent model retraining, so that the number of training samples that can be used in model retraining is greatly increased compared to pre-training, and the classification accuracy and generalization of the classification model after model retraining can be further improved.
[0049] Figure 4 FIG. 2 shows a schematic diagram of marking a semi-labeled sample according to an embodiment of the present disclosure. Figure 4 In the example shown, in step S105, the classification model pre-trained in step S103 can be used to mark the missing classification labels of the semi-labeled samples. Figure 3 For sample A in the pre-trained classification model, the classifier 2 can be used. The classification result (i.e., prediction result) of sample A, for example, classification result 6, is used as the missing classification label a2 of sample A, that is, the classification label of sample A corresponding to the category of classifier 2.
[0050] In some embodiments, after step S105, the classification model can be retrained (or re-trained) based on the semi-label samples in the second batch of samples after adding the missing classification labels. In different embodiments, the samples of the retrained classification model can include not only the semi-label samples with missing classification labels added, but also the full-label samples. Therefore, in one embodiment, the classification model can be retrained based on the third batch of samples, and the third batch of samples includes: the semi-label samples in the second batch of samples after adding the missing classification labels, and the full-label samples. In different embodiments, the full-label samples based on which the classification model is retrained may include the full-label samples based on the pre-training, or may include the full-label samples not used in the pre-training, and this specification does not limit this. In the above manner, the semi-label samples with missing classification labels and the full-label samples can be added in the retraining by using step S105, so that the number of training samples used for retraining is greatly increased, and the classification accuracy and generalization of the classification model after the model is retrained are further improved.
[0051] Figure 6 A schematic block diagram of a device for adding classification labels through a classification model according to another embodiment of the present disclosure is shown. The device is used to perform the following steps: Figure 1 The method shown in the figure includes a predetermined number of classifiers corresponding to a predetermined number of categories, such as Figure 6 As shown, the device 600 includes:
[0052] An acquisition unit 601 is configured to acquire a first batch of samples, where the first batch of samples includes a plurality of semi-labeled samples, and the semi-labeled samples have classification labels of some categories among a predetermined number of categories;
[0053] The model updating unit 602 is configured to respectively extract a plurality of first sample features corresponding to the plurality of half-labeled samples through a feature extraction network, and input each first sample feature into a portion of the predetermined number of classifiers to obtain a plurality of classification results corresponding to each half-labeled sample, wherein the portion of the classifiers respectively corresponds to a portion of the categories with classification labels of each half-labeled sample; determine a classification loss according to the plurality of classification results corresponding to each half-labeled sample and the classification labels of the portion of the categories of each half-labeled sample; and update the predetermined number of classifiers according to the classification loss;
[0054] The label adding unit 603 is configured to add missing classification labels to the semi-labeled samples included in the second batch of samples based on the updated predetermined number of classifiers.
[0055] The present disclosure also provides an electronic device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the following is achieved: Figure 1 The method shown.
[0056] You can also refer to the following Figure 7 , which shows a structural schematic diagram of an electronic device 800 suitable for implementing an embodiment of the present application. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0057] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701. The above-mentioned processing device 701 may be a general-purpose processor, a digital signal processor (Digital Signal Processing, DSP), a microprocessor or a microcontroller, and may further include an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage device 708 to the random access memory (RAM) 703. In RAM703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, ROM702 and RAM703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0058] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 7 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0059] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method for adding classification labels by a classification model provided in an embodiment of the present application are executed.
[0060] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the following steps provided in the present disclosure: Figure 1 The method of adding classification labels through the classification model shown in . Figure 8 A schematic diagram of a storage medium for implementing an embodiment of the present application. Figure 8 As shown, the storage medium 800 may be a non-transitory computer-readable storage medium for storing non-transitory computer-executable instructions 801. When the non-transitory computer-executable instructions 801 are executed by the processor, the method for adding classification labels through a classification model provided in the embodiment of the present application may be implemented. For example, when the non-transitory computer-executable instructions 801 are executed by the processor, one or more steps in the method for adding classification labels through a classification model provided in the embodiment of the present application may be executed. For example, the storage medium 800 may be applied to the above-mentioned electronic device. For example, the storage medium 800 may include a memory in the electronic device. For the description of the storage medium 800, reference may be made to the description of the memory in the embodiment of the electronic device, and the repeated parts will not be repeated. The specific functions and technical effects of the storage medium 800 may refer to the description of the method for adding classification labels through a classification model provided in the embodiment of the present application, and will not be repeated here.
[0061] It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a memory card of a smart phone, a storage component of a tablet computer, a portable computer disk, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0062] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the server, the electronic device implements the method for adding classification labels through a classification model provided in the embodiment of the present application.
[0063] Computer program code for performing the operations of the disclosed embodiments may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0064] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions. The units involved in the embodiments described in the present disclosure can be implemented by software or by hardware. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances. The functions described above in this article can be performed at least in part by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0065] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0066] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned public concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) to form a technical solution. In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of a single embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination.
[0067] The specific implementation methods described above further describe in detail the purpose, technical solutions and beneficial effects of the embodiments of the present invention. Although the subject matter has been described in a language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. It should be understood that the above is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for adding classification labels through a classification model, wherein the classification model includes a predetermined number of classifiers corresponding to a predetermined number of categories, the method comprising: Acquire a first batch of samples, wherein the first batch of samples includes a plurality of semi-labeled samples, and the semi-labeled samples have classification labels of some categories among the predetermined number of categories; Extracting a plurality of first sample features corresponding to the plurality of half-labeled samples respectively through a feature extraction network, inputting each first sample feature into a portion of the predetermined number of classifiers, and obtaining a plurality of classification results corresponding to each half-labeled sample, wherein the portion of the classifiers respectively corresponds to a portion of the categories with classification labels of each half-labeled sample; determining a classification loss according to the plurality of classification results corresponding to each half-labeled sample and the classification labels of the portion of the categories of each half-labeled sample; and updating the predetermined number of classifiers according to the classification loss; Based on the updated predetermined number of classifiers, missing classification labels are added to the semi-labeled samples included in the second batch of samples.
2. The method according to claim 1, further comprising: The classification model is retrained based on the semi-labeled samples in the second batch of samples after adding the missing classification labels.
3. The method according to claim 1, wherein: The first batch of samples also includes a plurality of full-label samples, wherein the full-label samples have classification labels of a predetermined number of categories; The method further comprises: Extracting multiple second sample features corresponding to the multiple full-label samples respectively through a feature extraction network, inputting each second sample feature into the predetermined number of classifiers respectively, and obtaining the predetermined number of classification results corresponding to each full-label sample; The determining of the classification loss according to the multiple classification results corresponding to each semi-labeled sample and the classification labels of some categories possessed by each semi-labeled sample includes: The classification loss is determined based on multiple classification results corresponding to each half-label sample, classification labels of some categories possessed by each half-label sample, and a predetermined number of classification results corresponding to each full-label sample, and a predetermined number of classification labels possessed by each full-label sample.
4. The method according to claim 1, wherein: Adding missing classification labels to the semi-labeled samples included in the second batch of samples based on the updated predetermined number of classifiers, comprising: Extracting third sample features corresponding to the semi-labeled samples included in the second batch of samples respectively through the feature extraction network, inputting the third sample features into the predetermined number of classifiers respectively, and obtaining multiple classification results corresponding to the semi-labeled samples included in the second batch of samples; According to a plurality of classification results corresponding to the semi-labeled samples included in the second batch of samples, missing classification labels are added to the semi-labeled samples included in the second batch of samples.
5. The method according to claim 2, wherein: Retraining the classification model based on the semi-labeled samples in the second batch of samples after adding the missing classification labels; The classification model is retrained based on a third batch of samples, where the third batch of samples includes: semi-labeled samples in the second batch of samples after adding missing classification labels, and full-labeled samples.
6. The method according to claim 1, wherein: The samples included in the first batch of samples are media data samples, and the media data samples include multiple modal data, and the multiple modal data include two or more of visual modal data, text modal data, motion modal data and audio modal data.
7. The method according to claim 1, wherein: The semi-labeled sample includes a plurality of modal data; the feature extraction network includes a plurality of sub-feature extraction networks; and the extracting a plurality of first sample features corresponding to the plurality of semi-labeled samples respectively through the feature extraction network includes: For any one of the multiple semi-labeled samples, multiple sub-modal features corresponding to the multiple modal data included in the any one semi-labeled sample are extracted respectively through multiple sub-feature extraction networks; and the multiple sub-modal features are fused to obtain a first sample feature corresponding to the any one semi-labeled sample.
8. The method according to claim 7, wherein: The fusing of the multiple sub-modal features includes: fusing the multiple sub-modal features through addition or cascade operation.
9. The method according to claim 1, wherein: The classifier includes a plurality of fully connected layers, and the classification loss is obtained by a cross entropy loss function; The classification model further includes the feature extraction network, and updating the predetermined number of classifiers according to the classification loss includes: According to the classification loss, the predetermined number of classifiers and the feature extraction network are updated.
10. A device for adding classification labels through a classification model, wherein the classification model includes a predetermined number of classifiers corresponding to a predetermined number of categories, the device comprising: An acquisition unit is configured to acquire a first batch of samples, wherein the first batch of samples includes a plurality of semi-labeled samples, and the semi-labeled samples have classification labels of some categories among a predetermined number of categories; The model updating unit is configured to respectively extract a plurality of first sample features corresponding to the plurality of half-labeled samples through a feature extraction network, respectively input each first sample feature into a portion of the predetermined number of classifiers to obtain a plurality of classification results corresponding to each half-labeled sample, wherein the portion of the classifiers respectively corresponds to a portion of the categories with classification labels of each half-labeled sample; determine a classification loss according to the plurality of classification results corresponding to each half-labeled sample and the classification labels of the portion of the categories of each half-labeled sample; and update the predetermined number of classifiers according to the classification loss; The label adding unit is configured to add missing classification labels to the semi-labeled samples included in the second batch of samples based on the updated predetermined number of classifiers.
11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 9.
12. An electronic device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 9 is implemented.