Data classification method and multi-classification model training method, device, equipment and medium

Abstract

The application provides a data classification method and a multi-classification model training method, device, equipment and medium, and is applied to the field of artificial intelligence. The data classification method comprises the following steps: obtaining to-be-classified data; inputting the to-be-classified data into a first model and a second model of a multi-classification model respectively to obtain a classification result of the to-be-classified data; the first model is obtained through a first training set, and the first training set comprises a plurality of first sample data carrying real classification labels; the second model is obtained through a second training set, and the second training set comprises a plurality of second sample data carrying a bias label set, and the bias label set comprises at least one candidate classification label. Through the data classification method provided by the application, the accuracy of data classification can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a data classification method, a multi-classification model training method, a device, equipment and a computer readable storage medium. BACKGROUND

[0002] Artificial intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0003] In various application scenarios, data classification has always been the focus of artificial intelligence. The traditional data classification scheme has low classification accuracy for the data to be classified, and it is difficult to meet the needs of actual scenarios. SUMMARY

[0004] The embodiments of the present application provide a data classification method, a multi-classification model training method, a device, equipment and a medium, which can improve the data classification accuracy.

[0005] The technical scheme of the embodiments of the present application is as follows:

[0006] The embodiments of the present application provide a data classification method, comprising: obtaining data to be classified; inputting the data to be classified into a first model and a second model of a multi-classification model respectively, to obtain a classification result of the data to be classified; wherein the first model is obtained through a first training set, and the first training set comprises a plurality of first sample data carrying real classification labels; the second model is obtained through a second training set, and the second training set comprises a plurality of second sample data carrying a bias label set, and the bias label set comprises at least one candidate classification label.

[0007] In some embodiments of the present application, there is at least one newly added first sample data in the plurality of first sample data; the real classification label carried by the newly added first sample data is obtained according to the second model; and / or there is at least one updated second sample data in the plurality of second sample data; the bias label set carried by the updated second sample data is obtained according to the first model.

[0008] In some embodiments of the present application, the inputting the to-be-classified data into the first model and the second model of the multi-classification model respectively to obtain a classification result of the to-be-classified data comprises: inputting the to-be-classified data into the first model to obtain a first classification result output by the first model; the first classification result comprises a first confidence degree corresponding to each of a plurality of preset categories; inputting the to-be-classified data into the second model to obtain a second classification result output by the second model; the second classification result comprises a second confidence degree corresponding to each of the plurality of preset categories; and determining the classification result of the to-be-classified data based on the first confidence degree and the second confidence degree corresponding to each of the preset categories.

[0009] In some embodiments of the present application, the determining the classification result of the to-be-classified data based on the first confidence degree and the second confidence degree corresponding to each of the preset categories comprises: obtaining a first precision parameter corresponding to the first classification result and a second precision parameter corresponding to the second classification result; the first precision parameter is used to represent the prediction precision of the first model; the second precision parameter is used to represent the prediction precision of the second model; determining a third confidence degree corresponding to each of the preset categories based on the first confidence degree and the second confidence degree corresponding to each of the preset categories, the first precision parameter and the second precision parameter; and taking the preset category corresponding to the maximum third confidence degree as the classification result of the to-be-classified data.

[0010] In some embodiments of the present application, the training process of the multi-classification model comprises: obtaining the first model through the first training set; obtaining the second model through the second training set; performing a training set updating operation to obtain an updated training set in the case that a stop condition is not met, iteratively training the multi-classification model by using the updated training set until the stop condition is reached, and outputting the trained multi-classification model; and the training set updating operation comprises at least one of the following: updating the first training set by using the second model, and updating the second training set by using the first model.

[0011] Embodiments of the present application provide a multi-classification model training method, comprising: obtaining a first model in the multi-classification model through a first training set, the first training set comprising a plurality of first sample data carrying real classification labels; obtaining a second model in the multi-classification model through a second training set, the second training set comprising a plurality of second sample data carrying a bias label set, the bias label set comprising at least one candidate classification label; performing a training set updating operation to obtain an updated training set in the case that a stop condition is not met, iteratively training the multi-classification model by using the updated training set until the stop condition is reached, and outputting the trained multi-classification model; and the training set updating operation comprises at least one of the following: updating the first training set by using the second model, and updating the second training set by using the first model.

[0012] In some embodiments of the present application, the method further comprises: constructing a first training set and a third training set based on the original data set; the third training set comprises a plurality of third sample data without classification labels; generating a bias label set corresponding to each third sample data according to each first sample data in the first training set; adding the corresponding bias label set to each third sample data to obtain the second training set.

[0013] In some embodiments of the present application, the constructing a first training set and a third training set based on the original data set comprises: obtaining a plurality of original data in the original data set; performing feature extraction on each original data to obtain original features corresponding to each original data and construct an original training set; extracting at least one first original feature and at least one second original feature from the original training set; constructing the first training set comprising a plurality of first sample data based on the at least one first original feature and the real classification label corresponding to each original feature; constructing the third training set based on the at least one second original feature.

[0014] In some embodiments of the present application, the original data can be at least one of the following: voice data, text data, image data, and video data.

[0015] In some embodiments of the present application, the generating a bias label set corresponding to each third sample data according to each first sample data in the first training set comprises: generating a weight matrix according to the similarity between each third sample data and each first sample data; generating a category confidence vector corresponding to each third sample data based on the weight matrix; the category confidence vector comprises a fourth confidence corresponding to each preset category; generating a bias label set corresponding to each third sample data according to the category confidence vector corresponding to each third sample data.

[0016] In some embodiments of the present application, the generating a weight matrix according to the similarity between each third sample data and each first sample data comprises: generating a weight vector corresponding to each third sample data according to the similarity between each third sample data and each first sample data; generating the weight matrix according to the weight vector corresponding to each third sample data.

[0017] In some embodiments of the present application, the generating the weight vector corresponding to each third sample data according to the similarity between each third sample data and each first sample data comprises: determining a near neighbor sample set of the third sample data in each first sample data; the near neighbor sample set comprises at least one near neighbor sample data; the near neighbor sample data is a first sample data similar to the third sample data; using a first weight generation rule to generate a weight element corresponding to the first sample data in the case that the first sample data is in the near neighbor sample set; using a second weight generation rule to generate a weight element corresponding to the first sample data in the case that the first sample data is not in the near neighbor sample set; and generating the weight vector corresponding to the third sample data based on the weight element corresponding to each sample data.

[0018] In some embodiments of the present application, the generating the class confidence vector corresponding to each third sample data based on the weight matrix comprises: determining a fourth confidence of the third sample data in each preset class according to each first sample data; and determining the class confidence vector corresponding to the third sample data according to the fourth confidence of the third sample data in each preset class.

[0019] In some embodiments of the present application, the generating the bias label set corresponding to each third sample data according to the class confidence vector corresponding to each third sample data comprises: performing normalization processing on the class confidence vector corresponding to the third sample data to obtain a normalized class confidence vector; the normalized class confidence vector comprises a fifth confidence of each preset class; and determining the bias label set corresponding to the third sample data according to the fifth confidence of each preset class.

[0020] In some embodiments of the present application, the updating the second training set through the first model comprises: obtaining at least one to-be-updated sample data in the third training set; inputting each to-be-updated sample data into the first model to obtain a to-be-updated bias label set corresponding to each to-be-updated sample data; and updating the second training set according to the to-be-updated bias label set corresponding to each to-be-updated sample data.

[0021] In some embodiments of the present application, the inputting each to-be-updated sample data into the first model to obtain a to-be-updated bias label set corresponding to each to-be-updated sample data comprises: inputting the to-be-updated sample data into the first model to obtain a third classification result output by the first model; inputting the to-be-updated sample data into the second model to obtain a fourth classification result output by the second model; and determining the to-be-updated bias label set corresponding to the to-be-updated sample data according to the third classification result and the fourth classification result.

[0022] In some embodiments of the present application, the third classification result includes a sixth confidence degree corresponding to each preset category in the plurality of preset categories; the fourth classification result includes a seventh confidence degree corresponding to each preset category; and the determining of the to-be-updated bias label set corresponding to the to-be-updated sample data according to the first classification result and the second classification result comprises: weighting and summing the sixth confidence degree and the seventh confidence degree corresponding to each preset category based on a preset balance parameter to obtain an eighth confidence degree corresponding to each preset category; and determining the to-be-updated bias label set corresponding to the to-be-updated sample data according to the eighth confidence degree of each preset category.

[0023] In some embodiments of the present application, the updating of the second training set according to the to-be-updated bias label set corresponding to each to-be-updated sample data comprises: querying to-be-updated second sample data corresponding to the to-be-updated sample data in the second training set; and updating the bias label set carried by the to-be-updated second sample data by using the to-be-updated bias label set.

[0024] In some embodiments of the present application, the updating of the first training set by the second model comprises: obtaining at least one to-be-labeled sample data in the third training set; inputting each to-be-labeled sample data into the second model to obtain a real classification label corresponding to each to-be-labeled sample data; and updating the first training set according to the real classification label corresponding to each to-be-labeled sample data.

[0025] In some embodiments of the present application, the inputting of each to-be-labeled sample data into the second model to obtain a real classification label corresponding to each to-be-labeled sample data comprises: inputting the to-be-labeled sample data into the second model to obtain a fifth classification result output by the second model; and determining the real classification label corresponding to the to-be-labeled sample data according to the fifth classification result.

[0026] In some embodiments of the present application, the fifth classification result includes a ninth confidence degree corresponding to each preset category in the plurality of preset categories; and the determining of the real classification label corresponding to the to-be-labeled sample data according to the fifth classification result comprises: taking a preset category corresponding to a maximum ninth confidence degree as the real classification label corresponding to the to-be-labeled sample data.

[0027] In some embodiments of the present application, the updating of the first training set according to the real classification label corresponding to each to-be-labeled sample data comprises: adding the real classification label corresponding to the to-be-labeled sample data; and adding the to-be-labeled sample data with the added real classification label to the first training set as new first sample data.

[0028] In some embodiments of the present application, the stopping condition comprises at least one of the following: the multi-classification model tends to converge; and a maximum iteration number is reached.

[0029] The embodiment of the present application provides a data classification device, and the device comprises:

[0030] An acquisition module is configured to acquire to-be-classified data.

[0031] A classification module is configured to input the to-be-classified data into a first model and a second model of a multi-classification model respectively, and obtain a classification result of the to-be-classified data; the first model is obtained through a first training set, and the first training set comprises a plurality of first sample data carrying real classification labels; the second model is obtained through a second training set, and the second training set comprises a plurality of second sample data carrying a bias label set, and the bias label set comprises at least one candidate classification label.

[0032] The embodiment of the present application provides a multi-classification model training device, and the device comprises:

[0033] A first model training module is configured to obtain a first model in the multi-classification model through a first training set, and the first training set comprises a plurality of first sample data carrying real classification labels.

[0034] A second model training module is configured to obtain a second model in the multi-classification model through a second training set, and the second training set comprises a plurality of second sample data carrying a bias label set, and the bias label set comprises at least one candidate classification label.

[0035] A third model training module is configured to perform a training set updating operation in a case where a stop condition is not met, obtain an updated training set, perform iterative training on the multi-classification model by using the updated training set, until the stop condition is reached, and output the trained multi-classification model; the training set updating operation comprises at least one of the following: updating the first training set through the second model, and updating the second training set through the first model.

[0036] The embodiment of the present application provides a computer device, and the device comprises:

[0037] A memory is configured to store executable instructions.

[0038] A processor is configured to execute the executable instructions stored in the memory, so as to implement the data classification method provided by the embodiment of the present application, or implement the multi-classification model training method provided by the embodiment of the present application.

[0039] The embodiment of the present application provides a computer readable storage medium, and the storage medium stores executable instructions, and is configured to cause a processor to execute the executable instructions, so as to implement the data classification method provided by the embodiment of the present application, or implement the multi-classification model training method provided by the embodiment of the present application.

[0040] The embodiment of the present application has the following beneficial effects:

[0041] The embodiments of the present application utilize the first model obtained based on a plurality of first sample data carrying real classification labels and the second model obtained based on a plurality of second sample data carrying bias label sets to classify the data to be classified. Since the multi-classification model integrates the classification results of the first model and the classification results of the second model, the classification result output by the multi-classification model can be closer to the actual classification of the data to be classified. In this way, the accuracy of data classification can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is an optional architecture schematic diagram of a data classification / multi-classification model training system provided by the embodiments of the present application;

[0043] Figure 2A is a structural schematic diagram of a data classification device provided by the embodiments of the present application;

[0044] Figure 2B is a structural schematic diagram of a multi-classification model training device provided by the embodiments of the present application;

[0045] Figure 3 is an optional flow schematic diagram of a data classification method provided by the embodiments of the present application;

[0046] Figure 4 is an optional flow schematic diagram of a data classification method provided by the embodiments of the present application;

[0047] Figure 5 is an optional flow schematic diagram of a data classification method provided by the embodiments of the present application;

[0048] Figure 6 is an optional flow schematic diagram of a multi-classification model training method provided by the embodiments of the present application;

[0049] Figure 7 is an optional flow schematic diagram of a multi-classification model training method provided by the embodiments of the present application;

[0050] Figure 8 is an optional flow schematic diagram of a multi-classification model training method provided by the embodiments of the present application;

[0051] Figure 9 is an optional flow schematic diagram of a multi-classification model training method provided by the embodiments of the present application;

[0052] Figure 10 is an optional framework schematic diagram of a semi-supervised learning system provided by the embodiments of the present application;

[0053] Figure 11 is an optional training flow schematic diagram provided by the embodiments of the present application. DETAILED DESCRIPTION

[0054] In order to make the purposes, technical solutions and advantages of the present application clearer, the following further describes the present application in conjunction with the accompanying drawings, and the described embodiments should not be regarded as limitations to the present application. All other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0055] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0056] In the following description, the terms "first\second\third" are merely to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described in the present embodiment can be implemented in an order other than that illustrated or described in the present embodiment.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0058] The relevant data collection and processing in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0059] The scheme provided by the embodiments of the present application relates to artificial intelligence technology, which is specifically described as follows:

[0060] AI is to use digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0061] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other major directions. The machine learning technology involved in the embodiments of the present application.

[0062] Among them, machine learning (Machine Learning, ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a special study of how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and inductive learning.

[0063] Before further detailing the embodiments of the present application, the terms and terms involved in the embodiments of the present application are explained, and the terms and terms involved in the embodiments of the present application are applicable to the following explanations.

[0064] (1) Supervised learning: a method of constructing a prediction model by learning a large number of training samples, each of which consists of two parts, a feature vector describing an object and its corresponding explicit label.

[0065] (2) Semi-supervised learning: a kind of learning under weak supervision, learning to construct a prediction model from a small amount of labeled data and a large amount of unlabeled data, that is, in addition to the labeled data, automatically use unlabeled data to improve learning effect. The difference between supervised learning and semi-supervised learning is that only a small amount of training samples in semi-supervised learning contain explicit labels, and other training samples do not contain explicit labels. Among them, semi-supervised learning can be divided into direct learning and pure semi-supervised learning, the former is based on closed world assumption, only predicting unlabeled data in training data, the latter is based on open world assumption, test data is unknown, not unlabeled data of training data.

[0066] (3) Partially labeled learning: a kind of weakly supervised learning, a method of constructing a prediction model by learning a large number of training samples, each of which consists of two parts, a feature vector describing an object and its corresponding multiple candidate labels, of which only one candidate label can truly reflect the object, and other candidate labels are false labels. The difference between supervised learning and partially labeled learning is that the training data label information of partially labeled learning is not accurate, and the prediction model needs to be learned in a noisy context.

[0067] (4) KNN: k-Nearest Neighbors, a supervised machine learning method. The idea is: given a training dataset, for a new input instance, find the k nearest neighbors in the training dataset, and the majority of the k instances belong to a certain class, then the input instance is classified into this class.

[0068] (5) co-training: co-training, a semi-supervised learning method based on divergence. It uses the "compatibility and complementarity" of multiple views, assuming that the data has two sufficient and conditionally independent views, and trains a classifier on each view based on the labeled samples. Let each classifier select "most confident" unlabeled samples to assign pseudo-labels, and provide unlabeled samples to the other classifier as new labeled samples for training and updating.

[0069] (6) Support Vector Machine: a supervised machine learning method. The basic model is a linear classifier with maximum margin defined in the feature space, and the learning strategy is maximum margin, which can be formalized as a convex quadratic programming problem, which is equivalent to the minimization problem of the regularized hinge loss function. When the training data is linearly inseparable, use the kernel trick to map the input space to a high-dimensional space to make it linearly separable.

[0070] Referring to Figure 1 , Figure 1 is an optional architecture diagram of the data classification / multi-classification model training system 100 provided by the embodiments of the present application, to realize a data classification / multi-classification model training application, the terminal (exemplarily shows the terminal 400-1 and the terminal 400-2) connects the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two. Figure 1 It is also shown that the server 200 can be a server cluster, which includes servers 200-1 to 200-3. Similarly, the servers 200-1 to 200-3 can be physical machines or virtual machines constructed by using virtualization technologies (such as container technology and virtual machine technology). The embodiments of the present application do not limit this, and of course, a single server can also be used to provide services in the present embodiment.

[0071] Referring to Figure 2A , Figure 2A is a structure diagram of the data classification device 500 provided by the embodiments of the present application, Figure 2AThe illustrated data classification device 500 includes at least one processor 510, memory 550, at least one network interface 520, and a user interface 530. The various components of the data classification device 500 are coupled together by a bus system 540, which is shown by a bus subsystem 540 in FIG. 5. As will be appreciated by those skilled in the art, the bus system 540 is used to facilitate communication between the various components and typically includes at least one of a data bus, a control bus, and a state line to communicate one or more signals between the components. Figure 2A The various buses are shown as the bus system 540 in FIG. 5 for clarity.

[0072] Referring to Figure 2B , Figure 2B is a structural schematic diagram of a multi-classification model training device 600 provided by an embodiment of the present application, Figure 2B The illustrated multi-classification model training device 600 includes at least one processor 610, memory 650, at least one network interface 620, and a user interface 630. The various components of the multi-classification model training device 600 are coupled together by a bus system 640, which is shown by a bus subsystem 640 in FIG. 6. As will be appreciated by those skilled in the art, the bus system 640 is used to facilitate communication between the various components and typically includes at least one of a data bus, a control bus, and a state line to communicate one or more signals between the components. Figure 2B The various buses are shown as the bus system 640 in FIG. 6 for clarity.

[0073] The processor 510 / 610 can be an integrated circuit chip that has a processing capability of a signal, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor.

[0074] The user interface 530 / 630 includes one or more output devices 531 / 631 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 / 630 also includes one or more input devices 532 / 632, including user interface components that facilitate input by a user, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons and controls.

[0075] The memory 550 / 650 includes volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read only memory (ROM), and the volatile memory can be a random access memory (RAM). The memory 550 / 650 described in the embodiments of the present application is intended to include any suitable type of memory. The memory 550 / 650 can optionally include one or more storage devices that are physically separate from the processor 510 / 610.

[0076] In some embodiments of the present application, the memory 550 / 650 can store data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, which are exemplarily illustrated below.

[0077] The operating system 551 / 651 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks;

[0078] The network communication module 552 / 652 is used to communicate with other computing devices via one or more (wired or wireless) network interfaces 520 / 620, exemplary network interfaces 520 / 620 including Bluetooth, wireless compatibility certification (WiFi), and universal serial bus (USB), and the like;

[0079] The display module 553 / 653 is used to enable the presentation of information via one or more output devices 531 / 631 (e.g., display screens, speakers, and the like) associated with the user interface 530 / 630 (e.g., user interfaces for operating peripheral devices and displaying content and information);

[0080] The input processing module 554 / 654 is used to detect and interpret one or more user inputs or interactions from one or more input devices 532 / 632.

[0081] In some embodiments of the present application, the data classification device / multi-classification model training device provided by the embodiments of the present application can be implemented in a combination of software and hardware. As an example, the data classification device / multi-classification model training device provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data classification method / multi-classification model training method provided by the embodiments of the present application.

[0082] In some embodiments of the present application, the data classification device / multi-classification model training device provided by the embodiments of the present application can be implemented in a software manner, Figure 2A The data classification device 555 stored in the memory 550 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: an acquisition module 5551 and a classification module 5552. Figure 2B The multi-classification model training device 655 stored in the memory 650 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: a first model training module 6551, a second model training module 6552 and a third model training module 6553, which are logical, and thus can be combined or further split according to the implemented functions

[0083] The functions of the various modules will be described below.

[0084] In some other embodiments, the device provided by the embodiments of the present application can be implemented in a hardware manner. As an example, the device provided by the embodiments of the present application can be a hardware decoding processor programmed to execute the data classification / multi-classification model training method provided by the embodiments of the present application. For example, the hardware decoding processor can use one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic elements.

[0085] The exemplary application and implementation of the server provided by the embodiments of the present application will be described in the embodiments of the present application, in which the data classification / multi-classification model training method provided by the embodiments of the present application will be described with the server as the execution subject.

[0086] Referring to Figure 3 , Figure 3 is an optional flowchart of the data classification method provided by the embodiments of the present application, which will be described in combination with Figure 3 the steps shown.

[0087] In step 301, the data to be classified is acquired.

[0088] In some embodiments of the present application, the data classification method provided by the embodiments of the present application can be applied to different application scenarios, and the data to be classified is different for different application scenarios. Therefore, the embodiments of the present application will be explained and described in multiple application scenarios.

[0089] For example, when the data classification method provided by the embodiments of the present application is applied to an image classification scenario, the data to be classified can include at least one image; when the data classification method provided by the embodiments of the present application is applied to a speech recognition scenario, the data to be classified can include at least one piece of audio data; and when the data classification method provided by the embodiments of the present application is applied to a machine translation scenario, the data to be classified can include at least one piece of text data. The above is the related content of multiple application scenarios provided by the embodiments of the present application. It should be noted that the data classification provided by the embodiments of the present application can not only be applied to image classification, speech recognition, machine translation and other application scenarios, but also can be applied to other application scenarios, which are not limited by the embodiments of the present application.

[0090] In step 302, the data to be classified is input into the first model and the second model of the multi-classification model respectively, and the classification result of the data to be classified is obtained; wherein the first model is obtained by a first training set, and the first training set includes a plurality of first sample data carrying real classification labels; the second model is obtained by a second training set, and the second training set includes a plurality of second sample data carrying a bias label set, and the bias label set includes at least one candidate classification label.

[0091] In some embodiments of the present application, the multi-classification model can include a first model and a second model, wherein the first model and the second model are used for classifying the data to be classified. After the data to be classified is input into the multi-classification model, the first classification result of the data to be classified can be obtained by the first model, and the second classification result of the data to be classified can also be obtained by the second model. The final classification result of the multi-classification model can be obtained by combining the first classification result and the second classification result.

[0092] In some embodiments of the present application, the first model is obtained by a first training set, wherein the first training set includes a plurality of first sample data carrying real classification labels, and the real classification labels are used to represent the actual classification result of the first sample data.

[0093] In some embodiments of the present application, the second model is obtained through a second training set, wherein the second training set comprises a plurality of second sample data carrying a bias label set, and the bias label set comprises at least one candidate classification label. For a second sample data, only one candidate classification label in the at least one candidate classification label corresponding to the second sample data is used to represent the actual classification result of the first sample data.

[0094] In some embodiments of the present application, there is at least one new first sample data in the plurality of first sample data; the real classification label carried by the new first sample data is obtained according to the second model; and / or there is at least one updated second sample data in the plurality of second sample data; the bias label set carried by the updated second sample data is obtained according to the first model.

[0095] In the training process of the multi-classification model, i.e. in the process of one iteration training, the first training set is updated according to the intermediate second model obtained in the last iteration training, i.e. at least one new first sample data is obtained, and the new first sample data is added to the first training set. The updated first training set can be used for the next iteration training to obtain a new first model.

[0096] In the training process of the multi-classification model, i.e. in the process of one iteration training, the second training set is updated according to the intermediate first model obtained in the last iteration training, i.e. at least one updated second sample data is obtained, and the bias label set carried by the updated second sample data is obtained according to the intermediate first model. The updated second training set can be used for the next iteration training to obtain a new second model.

[0097] It should be noted that in the process of one iteration training, the update of the first training set can be completed only by using the second model obtained in the last iteration training to obtain a new first model; the update of the second training set can be completed only by using the first model obtained in the last iteration to obtain a new second model; or the update of the first training set can be completed by using the second model obtained in the last iteration training, and the update of the second training set can be completed by using the first model obtained in the last iteration to obtain a new first model and a new second model.

[0098] Through the embodiments of the present application, the following advantages can be achieved Figure 3According to the above exemplary embodiments, the embodiments of the present application utilize the first model trained based on the first sample data carrying the real classification labels and the second model obtained based on the second sample data carrying the bias label set to classify the to-be-classified data. Since the multi-classification model integrates the classification results of the first model and the classification results of the second model, the classification result output by the multi-classification model can be closer to the actual classification of the to-be-classified data. In this way, the accuracy of data classification can be improved.

[0099] Referring to Figure 4 , Figure 4 is an optional flow diagram of a data classification method provided by the embodiments of the present application, based on the above Figure 3 , Figure 3 Step 302 in the above method can include steps 401 to 403, which will be described in combination with the steps shown in Figure 4

[0100] In step 401, the to-be-classified data is input into the first model to obtain a first classification result output by the first model; the first classification result includes a first confidence degree corresponding to each of a plurality of preset categories.

[0101] In some embodiments of the present application, after the to-be-classified data is input into the first model, a first classification result output by the first model can be obtained, which can include a first confidence degree corresponding to each of a plurality of preset categories. In some embodiments, the sum of the first confidence degrees of each preset category is 1. For example, if there are a plurality of preset categories A, B and C, the first classification result can include a first confidence degree A1 corresponding to the A category, a first confidence degree B1 corresponding to the B category and a first confidence degree C1 corresponding to the C category, wherein the sum of A1, B1 and C1 is 1.

[0102] In step 402, the to-be-classified data is input into the second model to obtain a second classification result output by the second model; the second classification result includes a second confidence degree corresponding to each of a plurality of preset categories.

[0103] In some embodiments of the present application, after the to-be-classified data is input into the second model, a second classification result output by the second model can be obtained, which can include a second confidence degree corresponding to each of a plurality of preset categories. In some embodiments, the sum of the second confidence degrees of each preset category is 1. For example, if there are a plurality of preset categories A, B and C, the second classification result can include a second confidence degree A2 corresponding to the A category, a second confidence degree B2 corresponding to the B category and a second confidence degree C2 corresponding to the C category, wherein the sum of A2, B2 and C2 is 1.

[0104] ​In step 403, a classification result of the data to be classified is determined based on the first confidence and the second confidence corresponding to each preset category.

[0105] In some embodiments of the present application, step 403 can be implemented in the following way:

[0106] In step 4031, a first precision parameter corresponding to the first classification result and a second precision parameter corresponding to the second classification result are obtained; the first precision parameter is used to represent the prediction precision of the first model; and the second precision parameter is used to represent the prediction precision of the second model.

[0107] The precision parameter of each model is used to represent the closeness between the classification result of the data to be classified output by the model and the actual classification result of the data to be classified.

[0108] In step 4032, a third confidence corresponding to each preset category is determined based on the first confidence and the second confidence corresponding to each preset category, the first precision parameter and the second precision parameter.

[0109] The first precision parameter is taken as the weight of the first confidence of each preset category, the second precision parameter is taken as the weight of the second confidence of each preset category, the first confidence and the second confidence of each preset category are weighted and summed to obtain the third confidence corresponding to each preset category. The third confidence of one preset category is the sum of the product of the first confidence and the first precision parameter and the product of the second confidence and the second precision parameter.

[0110] For example, based on the above example, if there is a first precision parameter J1, a second precision parameter J2, a first confidence A1 corresponding to the A category, and a second confidence A2 corresponding to the A category, the third confidence A3 corresponding to the A category is obtained (A3=A1×J1+A2×J2).

[0111] In step 4033, the preset category corresponding to the maximum third confidence is taken as the classification result of the data to be classified.

[0112] From the above example implementation of the present application, Figure 4 As can be seen from the above example implementation of the present application, by obtaining the first precision parameter corresponding to the first model and the second precision parameter corresponding to the second model, and weighting the first classification result output by the first model by the first precision parameter and weighting the second classification result by the second precision parameter, the final classification result not only includes the classification results of the two classification models, but also combines the classification precision of each classification model. In this way, the obtained classification result can be more accurate.

[0113] Referring to Figure 5 , Figure 5is an optional flowchart of the data classification method provided by the embodiment of the present application, based on the above embodiment, taking Figure 3 for example, Figure 3 The training process of the multi-classification model can include steps 501 to 503, which will be described in combination with the steps shown in the following figures. Figure 5

[0114] In step 501, a first model is obtained by a first training set.

[0115] In step 502, a second model is obtained by a second training set.

[0116] In some embodiments of the present application, the first training set and the second training set can be obtained in the database, the first model is obtained by training the preset initial first model using the first training set, and the second model is obtained by training the preset initial second model using the second training set.

[0117] In some embodiments of the present application, after obtaining the first model and the second model, it is determined whether the first model and the second model obtained at this time meet the stopping condition, and after the stopping condition is met, the multi-classification model is generated according to the first model and the second model at this time, the multi-classification model includes the first model and the second model, and the final classification result is generated according to the first classification result output by the first model and the second classification result output by the second model.

[0118] In step 503, in the case where the stopping condition is not met, a training set updating operation is performed to obtain an updated training set, and the multi-classification model is iteratively trained using the updated training set until the stopping condition is met, and the trained multi-classification model is output; the training set updating operation includes at least one of the following: updating the first training set by the second model, and updating the second training set by the first model.

[0119] In some embodiments of the present application, in the case where the stopping condition is not met, one iteration training is performed, and whether the stopping condition is met after the iteration training is continued, in the case where the stopping condition is met, the multi-classification model is output, and in the case where the stopping condition is not met, the iteration training is continued.

[0120] In some embodiments of the present application, in the case where the stopping condition is not met, one iteration training is performed, and whether the stopping condition is met after the iteration training is continued, in the case where the stopping condition is met, the multi-classification model is output, and in the case where the stopping condition is not met, the iteration training is continued.

[0121] ​In a case where the training set updating operation comprises updating the first training set by the second model and updating the second training set by the first model, the iterative training of the multi-classification model by using the updated training sets comprises training the first model by using the updated first training set and training the second model by using the updated second training set. In a case where the training set updating operation comprises updating the first training set by the second model, the iterative training of the multi-classification model by using the updated training sets comprises training the first model by using the updated first training set. In a case where the training set updating operation comprises updating the second training set by the first model, the iterative training of the multi-classification model by using the updated training sets comprises training the second model by using the updated second training set.

[0122] Through the above exemplary embodiments of the application, in the model training process, the first model is used to update the second training set for obtaining the second model, and the second model is used to update the first training set for obtaining the first model, so that the classification accuracy of the first model and the second model can be continuously improved in the process of completing the training information interaction between the first model and the second model, and the classification accuracy of the overall multi-classification model can be improved. Meanwhile, in the process of updating the first training set by the second model, the first sample data is mined, the workload of manual marking is greatly reduced, and the overall efficiency of data classification is improved. Meanwhile, the first model and the second model are used to update the training set, so that the convergence speed of the multi-classification model can be accelerated and the model training efficiency can be improved. Figure 5 Referring to

[0123] , Figure 6 , Figure 6 is an optional flowchart of the multi-classification model training method provided by the embodiments of the application, which will be described in combination with the steps shown in Figure 6 .

[0124] In step 601, a first model in a multi-classification model is obtained by a first training set, and the first training set comprises a plurality of first sample data carrying real classification labels.

[0125] In step 602, a second model in the multi-classification model is obtained by a second training set, and the second training set comprises a plurality of second sample data carrying a bias label set, and the bias label set comprises at least one candidate classification label.

[0126] In step 603, whether a stop condition is met.

[0127] In a case where the stop condition is not met, step 604 is performed, and in a case where the stop condition is met, step 606 is performed.

[0128] In step 604, a training set updating operation is performed to obtain an updated training set; the training set updating operation includes at least one of the following: updating the first training set by the second model, and updating the second training set by the first model.

[0129] In step 605, the multi-classification model is iteratively trained using the updated training set.

[0130] In some embodiments of the present application, when the training set updating operation includes updating the first training set by the second model and updating the second training set by the first model, the iteratively training the multi-classification model using the updated training set includes training the first model using the updated first training set and training the second model using the updated second training set. When the training set updating operation includes updating the first training set by the second model, the iteratively training the multi-classification model using the updated training set includes training the first model using the updated first training set. When the training set updating operation includes updating the second training set by the first model, the iteratively training the multi-classification model using the updated training set includes training the second model using the updated second training set.

[0131] In step 606, the trained multi-classification model is output.

[0132] In some embodiments of the present application, the trained multi-classification model includes the trained first model and the trained second model when the above-mentioned stopping condition is met, and the trained multi-classification model is further configured to generate a final classification result according to the first classification result output by the trained first model and the second classification result, and the method of generating the final classification result can refer to the method provided in the above-mentioned embodiments.

[0133] Through the above-mentioned exemplary embodiments of the present application, Figure 6 As can be seen from the above-mentioned exemplary embodiments of the present application, by updating the second training set used to obtain the second model by the first model and updating the first training set used to obtain the first model by the second model in the model training process, the classification accuracy of the first model and the second model can be continuously improved in the process of completing the training information interaction between the first model and the second model, and thus the classification accuracy of the overall multi-classification model can be improved. Meanwhile, in the present application, the first sample data is mined in the process of updating the first training set by the second model, which greatly reduces the workload of manual marking and improves the overall efficiency of data classification. Meanwhile, the present application accelerates the convergence speed of the multi-classification model and improves the model training efficiency by the way of updating the training set of the first model and the second model.

[0134] Referring to Figure 7 , Figure 7is an optional flowchart of a multi-classification model training method provided by an embodiment of the present application, based on the above embodiment, based on Figure 6 For example, the method further includes the steps of obtaining a first training set and a second training set, which will be described in combination with Figure 7 the steps shown.

[0135] In step 701, based on the original data set, a first training set and a third training set are constructed; the third training set includes a plurality of third sample data without classification labels.

[0136] Among them, the original data set includes a plurality of original data, and the original data includes at least one of the following: voice data, text data, image data and video data.

[0137] In some embodiments of the present application, the above step 701 can be implemented in the following way:

[0138] In step 7011, a plurality of original data in the original data set is obtained.

[0139] In step 7012, feature extraction is performed on each original data to obtain original features corresponding to each original data, and an original training set is constructed.

[0140] Among them, the original features can be embodied in the form of feature vectors, for example, the original features can be N-dimensional vectors.

[0141] In step 7013, at least one first original feature and at least one second original feature are extracted from the original training set.

[0142] Among them, the extraction can be performed according to a certain extraction rule. The extraction rule can include extraction according to proportion, extraction according to the size of the original feature, extraction according to the feature identifier of the original feature, etc. For example, extraction according to proportion, at least one original feature and at least one second original feature can be extracted according to the proportion of L1:L2, wherein L2 is much larger than L1, that is, the number of second original features obtained is much larger than the number of first original features.

[0143] In step 7014, based on at least one first original feature and the real classification label corresponding to each original feature, a first training set including a plurality of first sample data is constructed.

[0144] Among them, after obtaining at least one first original feature, the real classification label corresponding to each first original feature can be obtained respectively, and the first original feature with the real classification label added is added to the first training set as the first sample data, so as to construct the first training set including a plurality of first sample data.

[0145] In step 7015, a third training set is constructed based on at least one second original feature.

[0146] In this process, after obtaining at least one second original feature, each second original feature can be added to the second training set as second sample data to construct a second training set that includes multiple second sample data.

[0147] In some embodiments of this application, the raw data includes at least one of the following: voice data, text data, image data, and video data. When the raw data is voice data, the features extracted from the voice data can be raw voice features; when the raw data is text data, the features extracted from the text data can be raw text features; when the raw data is image data, the features extracted from the image data can be raw image features; and when the raw data is video data, the features extracted from the video data can be raw video features.

[0148] In some embodiments of this application, the obtained first training set can be represented as ,in, For d-dimensional feature vectors, For example The mark, For containing q The third training set, obtained from a set of labels for 3 preset categories, can be represented as: .

[0149] In step 702, a partial label set corresponding to each third sample data is generated based on each first sample data in the first training set.

[0150] In some embodiments of this application, step 702 can be implemented in the following ways:

[0151] In step 7021, a weight matrix is ​​generated based on the similarity between each third sample data and each first sample data.

[0152] In some embodiments of this application, step 7021 can be implemented as follows: generating a weight vector corresponding to each third sample data based on the similarity between each third sample data and each first sample data; generating a weight matrix based on the weight vector of each third sample data.

[0153] The generating of the weight vector corresponding to each third sample data according to the similarity between each third sample data and each first sample data comprises: determining a near neighbor sample set of the third sample data in each first sample data; the near neighbor sample set comprises at least one near neighbor sample data; the near neighbor sample data is a first sample data similar to the third sample data; using a first weight generating rule to generate a weight element corresponding to the first sample data in the case that the first sample data is in the near neighbor sample set; using a second weight generating rule to generate a weight element corresponding to the first sample data in the case that the first sample data is not in the near neighbor sample set; and generating the weight vector corresponding to the third sample data based on the weight element corresponding to each sample data.

[0154] In some embodiments of the present application, the obtained first training set can be represented as , and the obtained third training set can be represented as In the case that the obtained first training set can be represented as , the near neighbor sample set of each third sample data in the obtained first training set can be determined respectively . The near neighbor sample set of each third sample data can be obtained by determining the similarity between the third sample data and each first sample data, and taking the k first sample data with the highest similarity as the near neighbor sample set of the third sample data. In one of the embodiments, the KNN (k-Nearest Neighbor) algorithm can be used to find the near neighbor sample set of each third sample data in the first training set.

[0155] In some embodiments of the present application, for a third sample data, the weight vector corresponding to the third sample data comprises the weight element determined by the third sample data and each first sample data. That is, for the generated weight matrix W, the weight matrix W comprises the weight vectors corresponding to U third sample data , and the weight vector corresponding to each third sample data comprises the weight element determined by the third sample data and each first sample data .

[0156] For example, in the case that U=6 and L=4, the corresponding weight matrix can be represented as formula (1-1):

[0157] Formula (1-1);

[0158] In the case that U=6 and L=4, the weight vector corresponding to the third sample data can be represented as formula (1-2):​​​​​

[0159] Formula (1-2);

[0160] For any one third sample data, each weight element in the weight vector corresponding to the third sample data can be generated according to different weight generation rules according to whether the first sample data is in the neighbor sample set corresponding to the third sample data, that is, in the case where the first sample data is in the neighbor sample set, the first weight generation rule is used to generate the weight element corresponding to the first sample data; in the case where the first sample data is not in the neighbor sample set, the second weight generation rule is used to generate the weight element corresponding to the first sample data.

[0161] For example, the generation of the weight element can be realized by the weight generation rule shown in formula (1-3):

[0162] Formula (1-3);

[0163] wherein, in the case where the first sample data is in the neighbor sample set of the third sample data, the weight element determined by the first sample data and the third sample data is determined by the first sample data and the third sample data; in the case where the first sample data is not in the neighbor sample set, the parameter "0" is taken as the weight element determined by the first sample data and the third sample data.

[0164] wherein, is the bandwidth parameter of the kernel function, and is 1 by default; is the 2-norm of , that is, the Euclidean distance of two feature vectors. The bandwidth controls the local action range of the Gaussian kernel function, and the larger the bandwidth, the larger the influence range. The bandwidth here is essentially the variance. The embodiments of the present application can normalize the original data, and the variance of the processed data is 1. Therefore, the bandwidth parameter is 1 by default, and can be adjusted according to the variance of the training data or the model effect.

[0165] In step 7022, based on the weight matrix, a class confidence vector corresponding to each third sample data is generated; the class confidence vector includes a fourth confidence corresponding to each preset class.

[0166] In some embodiments of the present application, the above step 7022 can be realized by the following way: according to each first sample data, the fourth confidence of the third sample data in each preset class is determined; according to the fourth confidence of the third sample data in each preset class, the class confidence vector corresponding to the third sample data is determined.

[0167] The fourth confidence of each third sample data in each preset category can be determined by formula (1-4).

[0168] Formula (1-4);

[0169] wherein, identifying the third sample data the fourth confidence of the third sample data in the kth preset category, as an example the label of, the labeled data set in the neighbor of, in 1 if true, otherwise 0.

[0170] For example, based on the above example, for the third sample data , if the corresponding weight vector is , the fourth confidence of the third sample data in the kth preset category can be determined by formula (1-5).

[0171] Formula (1-5); In step 7023, a bias label set corresponding to each third sample data is generated according to the category confidence vector corresponding to each third sample data.

[0172] In some embodiments of the present application, for each third sample data, at least one preset category with a larger fourth confidence can be determined in order of the size of the fourth confidence, and the at least one preset category is taken as the bias label set corresponding to the third sample data. For example, for the third sample data

[0173] , if the fourth confidence of the third sample data in the kth preset category is , , , , in order of size, at least one preset category with a larger fourth confidence is determined, including , and , then the bias label set corresponding to the third sample data includes the second preset category, the fourth preset category and the fifth preset category.

[0174] In some embodiments of the present application, for each third sample data, at least one preset category greater than a preset fourth confidence threshold can be determined as the bias label set corresponding to the third sample data according to the fourth confidence threshold. For example, for the third sample data , if the fourth confidence of the third sample data in the kth preset category is the fourth confidence of the kth preset category is respectively , , ; according to a preset fourth confidence threshold , at least one preset category greater than the , including and , the third sample data corresponding to the bias label set includes the second preset category and the fifth preset category.

[0175] In some embodiments of the present application, the above step 7023 can also be implemented in the following manner: the category confidence vector corresponding to the third sample data is normalized to obtain a normalized category confidence vector; the normalized category confidence vector includes a fifth confidence of each preset category; according to the fifth confidence of each preset category, the bias label set corresponding to the third sample data is determined.

[0176] Wherein, the normalization processing of the category confidence vector corresponding to the third sample data can be implemented by the following formula (1-6):

[0177] Formula (1-6);

[0178] Wherein, represents the fifth confidence of the third sample data on the kth preset category after normalization processing.

[0179] Then, the bias label set corresponding to the third sample data can be represented as formula (1-7);

[0180] Formula (1-7);

[0181] Wherein, the bias label set corresponding to the third sample data .

[0182] In step 703, the corresponding bias label set is added to each third sample data to obtain a second training set.

[0183] In some embodiments of the present application, for each third sample data , the corresponding bias label set is added, then the corresponding second sample data can be obtained. Thus, the second training set including each second sample data can be obtained as .

[0184] Through the embodiments of the present application, for each third sample data Figure 7As can be known from the above exemplary embodiments, the embodiments of the present application generate the corresponding bias label set for each third sample data by obtaining the near neighbor sample set corresponding to each third sample data in the first training set. In this way, a large amount of second sample data carrying supervision information can be automatically generated under the premise of very few labeled data to obtain the second training set, and the second model under the semi-supervised learning method can be obtained through the second training set, thereby providing a data basis for the model precision of the subsequent overall multi-classification model. Meanwhile, since the second model is trained by using the second sample data carrying the bias label set including multiple candidate classification labels, compared with the semi-supervised learning method using only a single candidate label in the related art, the problem of label designation error caused by too few labeled samples and poor initial model performance and the subsequent decline in model performance can be solved, and the second model obtained through the embodiments of the present application can solve the multi-classification problem, and compared with the semi-supervised model that can only solve the binary classification problem in the related art, the performance of the second model obtained by the present application is more superior.

[0185] Referring to Figure 8 , Figure 8 is an optional flowchart of a multi-classification model training method provided by the embodiments of the present application, based on the above embodiments, taken Figure 6 as an example, Figure 6 the second training set is updated through the first model in the embodiments, which can be implemented through steps 801 to 803, which will be described in combination with the steps shown in Figure 8 .

[0186] In step 801, at least one sample data to be updated is obtained in the third training set.

[0187] In some embodiments of the present application, step 801 randomly obtains at least one third sample data from the third training set as the obtained at least one sample data to be updated in a replacement manner.

[0188] In step 802, each sample data to be updated is input into the first model and the second model to obtain the bias label set corresponding to each sample data to be updated.

[0189] In some embodiments of the present application, the above step 802 can be implemented in the following manner:

[0190] In step 8021, the sample data to be updated is input into the first model to obtain the third classification result output by the first model.

[0191] In some embodiments of the present application, the first model is a classification model learned from the first training set above, and the first model is represented as , where t is the number of iterations, and the first model The sample data to be updated corresponding third classification result, and satisfy The supervised learning algorithm used can be freely set, that is, by the supervised learning algorithm and the first training set The first model The support vector machine is used in the present application.

[0192] In step 8022, the sample data to be updated is input into the second model to obtain a fourth classification result output by the second model.

[0193] In some embodiments of the present application, the second model is a classification model learned from the second training set above, and the second model is represented as wherein, represents mapping to, t is the number of iterations, and the second model The sample data to be updated corresponding fourth classification result, and satisfy The partial label learning algorithm used can be freely set, that is, by the partial label learning algorithm and the second training set The second model The IPAL (Instance based PArtial Label Learning) is used in the present application.

[0194] In step 8023, the sample data to be updated corresponding to the set of partial labels to be updated is determined according to the third classification result and the fourth classification result.

[0195] wherein the third classification result includes a sixth confidence degree corresponding to each preset category; and the fourth classification result includes a seventh confidence degree corresponding to each preset category. The sample data to be updated corresponding to the set of partial labels to be updated is determined according to the third classification result and the fourth classification result, including: based on a preset balance parameter, performing weighted summation on the sixth confidence degree and the seventh confidence degree corresponding to each preset category to obtain an eighth confidence degree corresponding to each preset category; and determining the set of partial labels to be updated corresponding to the sample data to be updated according to the eighth confidence degree of each preset category.

[0196] In some embodiments of the present application, the eighth confidence degree corresponding to each k preset category can be determined by formula (1-8).

[0197] Formula (1-8);

[0198] wherein, denotes the third classification result output by the first model, denotes the fourth classification result output by the second model, denotes the balance parameter.

[0199] For example, for one to-be-updated sample data , there are multiple preset categories, A category, B category and C category, the sixth confidence corresponding to each preset category in the third classification result can be obtained, including A6, B6 and C6; the seventh confidence corresponding to each preset category in the fourth classification result can be obtained, including A7, B7 and C7. The eighth confidence A8 corresponding to the A category is ×A6+ (1- ) ×A7, and so on. The eighth confidence B8 and C8 corresponding to the B category and the C category can be obtained.

[0200] In some embodiments of the present application, the to-be-updated bias label set corresponding to the to-be-updated sample data can be determined by formula (1-9).

[0201] Formula (1-9);

[0202] wherein, is the to-be-updated bias label set corresponding to the to-be-updated sample data. The formula (1-9) is used to determine the preset category corresponding to the eighth confidence greater than the average confidence as one to-be-updated candidate classification label in the to-be-updated bias label set.

[0203] In step 803, the second training set is updated according to the to-be-updated bias label set corresponding to each to-be-updated sample data.

[0204] In some embodiments of the present application, the above step 803 can be implemented in the following way:

[0205] In step 8031, the to-be-updated second sample data corresponding to the to-be-updated sample data is queried in the second training set.

[0206] In step 8032, the bias label set carried by the to-be-updated second sample data is updated by using the to-be-updated bias label set.

[0207] In some embodiments of the present application, each to-be-updated sample data can be searched in the second training set, that is, by using the to-be-updated sample data The original second sample data is found in the second training set as At the same time, the obtained to-be-updated bias label set updates the original bias label set , the updated second sample data is obtained .

[0208] From the above exemplary implementation of the embodiment of the present application to Figure 8 , the embodiment of the present application updates the plurality of candidate classification labels carried by the second sample data through the first model, so that the updated second sample data can be more close to the actual classification result. In this way, the second model with higher prediction accuracy can be obtained through the updated second training set, which provides a basis for improving the overall model accuracy of the multi-classification model.

[0209] Referring to Figure 9 , Figure 9 is an optional flowchart of the multi-classification model training method provided by the embodiment of the present application, based on the above embodiment, taking Figure 6 as an example, Figure 6 the updating of the first training set through the second model in the embodiment can be implemented through steps 901 to 903, which will be described in combination with the steps shown in Figure 9 .

[0210] In step 901, at least one sample data to be labeled is obtained in the third training set.

[0211] In some embodiments of the present application, step 901 randomly obtains at least one third sample data from the third training set as the obtained at least one sample data to be labeled in a replacement manner.

[0212] In step 902, each sample data to be labeled is input into the second model to obtain a real classification label corresponding to each sample data to be labeled.

[0213] In some embodiments of the present application, the above step 902 can be implemented in the following manner:

[0214] In step 9021, the sample data to be labeled is input into the second model to obtain a fifth classification result output by the second model;

[0215] In some embodiments of the present application, the second model is a classification model learned from the above second training set , and the first model is represented as , wherein t is the number of iterations, and the second model can obtain the fifth classification result corresponding to the sample data to be labeled , and satisfies . The bias label learning algorithm used can be freely set, that is , and the IPAL is adopted in the present application.

[0216] In step 9022, a real classification label corresponding to the sample data to be labeled is determined according to the fifth classification result.

[0217] The fifth classification result includes a ninth confidence degree corresponding to each preset category in the plurality of preset categories. The real classification label corresponding to the sample data to be labeled is determined according to the fifth classification result, including: taking a preset category corresponding to a maximum ninth confidence degree as the real classification label corresponding to the sample data to be labeled.

[0218] In some embodiments of the present application, the real classification label corresponding to the sample data to be labeled can be determined by formula (1-10).

[0219] Formula (1-10);

[0220] In formula (1-10), In the case of , the maximum value of is obtained.

[0221] In step 903, the first training set is updated according to the real classification label corresponding to each sample data to be updated.

[0222] In some embodiments of the present application, the above step 903 can be implemented in the following manner:

[0223] In step 9031, the real classification label corresponding to the sample data to be labeled is added.

[0224] In step 9032, the sample data to be labeled to which the real classification label is added is added to the first training set as new first sample data.

[0225] In some embodiments of the present application, for the sample data to be labeled , the real classification label is added to the sample data to be labeled to obtain new first sample data ( ) and is added to the first training set. The number of first samples in the updated first training set is more than the number of first samples in the first training set before updating.

[0226] From the above exemplary implementation of the present application for Figure 9 , it can be seen that the present application increases new first sample data carrying real classification labels for the first training set through the second model. On the basis of the original small amount of first sample data, the number of first sample data is increased, so that the updated first training set is more abundant. In this way, the first model with higher prediction accuracy can be obtained through the updated first training set, which provides a basis for improving the overall model accuracy of the multi-classification model.

[0227] Below, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0228] The embodiments of the present application propose a new multi-class semi-supervised learning method, which is used for constructing a candidate label set for unlabelled data instead of a single pseudo-label designation. A learner is learned from the partial label samples containing the candidate label set (equivalent to the second model in the above embodiments), and another learner is learned from the labelled samples (equivalent to the first model in the above embodiments), and then the label information of the two learners is communicated to update the overall model (equivalent to the multi-class model in the above embodiments). Through iterative updating, the final learning model (trained multi-class model) is obtained, which can be used to predict the labels of unseen samples.

[0229] In the related art, semi-supervised learning methods can be divided into generative methods, semi-supervised support vector machines, graph semi-supervised learning, and divergence-based methods. Among them, the divergence-based method co-training is one of the most representative methods, two classifiers learn on two different views of the same data set, and each classifier selects the “most confident” unlabelled sample to assign a pseudo-label, and provides the pseudo-labelled sample to the other classifier as a newly added labelled sample for training and updating. The present application adopts a similar strategy as co-training, i.e., exchanging label information between two classifiers, but unlike co-training, the present application does not set two views for the example labels, but uses a unified feature, and learns to assign a candidate label set from unlabelled samples instead of a single label.

[0230] In the related art, semi-supervised support vector machines are also another very popular semi-supervised learning strategy. Without considering the labelled samples, the support vector machine tries to find the maximum interval dividing hyperplane, but after considering the unlabelled samples, the semi-supervised support vector machine tries to find a dividing hyperplane that can separate the labelled samples and pass through the low-density area of the data. It often considers various possible labels for unlabelled samples, such as trying to treat each unlabelled sample as a positive example or a negative example, and then finding a result among all the designations that maximizes the interval. However, the pseudo-label designation may be wrong, so the safe semi-supervised support vector machine technology is developed to ensure that the use of unlabelled data is beneficial.

[0231] Graph semi-supervised learning, such as label propagation, uses the graph structure contained in labelled and unlabelled samples to estimate the pseudo-labels of unlabelled samples, so the graph semi-supervised learning method is usually inductive learning, but the present application is a pure semi-supervised learning method, which has the ability to predict the labels of unseen examples.

[0232] Generative methods are those that are based directly on generative models. These methods assume that all data is "generated" by the same latent model, and the parameters of the latent model link the unlabeled data to the learning goal. The labels of the unlabeled data can be seen as missing parameters of the model, and can often be solved by maximum likelihood estimation based on the EM algorithm. The key of this method is that the assumed model must match the real data distribution, but it is difficult to make accurate assumptions in real tasks.

[0233] Partial label learning solves the problem of inaccurate supervision information of training samples, that is, each training sample corresponds to a candidate label set, but only one label in the set is the real label, and the other labels can be regarded as noise. The main method strategy is to eliminate the ambiguity of the candidate label set, which can be divided into recognition disambiguation and average disambiguation. For recognition disambiguation, the real label of the example is regarded as a hidden variable, and the target function with the embedded hidden variable is optimized by iteration to realize disambiguation. For average disambiguation, each candidate label of the partial label object is given the same weight, and disambiguation is realized by learning the output of the model on each candidate label. In addition to the disambiguation strategy, some methods also convert the partial label learning problem into other existing learning problems.

[0234] Then the applicant found through research that the existing semi-supervised learning scheme has the following disadvantages in practice:

[0235] (1) The semi-supervised support vector machine, graph semi-supervised learning and divergence-based method all specify a single pseudo-label for the unlabeled data, and then add the pseudo-label to the labeled data set as a real label to derive the model. However, the specification of a single pseudo-label is often prone to error, especially when the labeled data is scarce and the initial prediction model performs poorly, thus leading to a decline in the performance of the subsequent model.

[0236] (2) Divergence-based methods such as co-training require multi-view data, but there are few data sets that meet the conditions in real tasks, and some design is required.

[0237] (3) The graph semi-supervised learning method is usually inductive learning, based on the closed world assumption, and cannot predict the label of unseen samples, but only can predict the label of unlabeled data in the training set.

[0238] (4) The generative method relies on the accuracy of the model assumption, that is, the assumed generative model must match the real data distribution, otherwise the use of unlabeled data will reduce the generalization performance, but it is difficult to make an accurate model assumption in real tasks, and sufficient domain knowledge is required.

[0239] (5) Many semi-supervised learning methods can only solve the positive and negative binary classification problem, and the one-to-one or one-to-many strategy is required to integrate the binary classification model to solve the multi-classification problem.

[0240] To solve the above problems, the embodiment of the application provides a novel semi-supervised learning method. First, a candidate label set is constructed for unlabelled data instead of a single pseudo label designation. Then, a learner is learned from the partial label sample containing the candidate label set, and another learner is learned from the labelled sample. The label information of the two learners is communicated to update the overall model. Through iterative updating, the final learning model is obtained, which can be used to predict the label of unseen samples.

[0241] The semi-supervised learning method has the following characteristics: (1) Instead of a single label designation as in the previous semi-supervised learning method, a candidate label set is designated, and the true label is contained in the candidate label set. The single label designation is prone to errors and leads to a decrease in the performance of the subsequent derived model. The use of the candidate label set improves the robustness of the label designation to a certain extent and has a higher fault tolerance. (2) After the candidate label set is designated, a partial label learning model can be learned from the unlabelled data, and a supervised learning model can be learned from the labelled data. The integration and iterative updating of the two models can effectively improve the learning of the data in the semi-supervised situation. (3) The method is very flexible, and the partial label learning method and the supervised learning method are not limited. The optimal method can be selected according to the actual effect to achieve better performance.

[0242] The semi-supervised learning method provided by the application can be used to learn a model in a situation where labelled data is scarce and the cost of obtaining data labels is too high, especially in some professional fields such as finance, law and medicine. In these fields, data labelling requires professional personnel or even experts, and the cost is high. In addition, the method can be applied to multiple fields such as audio, image and text. Compared with the supervised method using only a small amount of labelled data, the method can significantly improve the accuracy of the model.

[0243] Taking text sentiment classification as an example, a large amount of text is obtained, and a large amount of manpower and material resources are required to obtain the label information. The application can only label a small amount of samples, extract text features by combining unlabelled samples, and automatically identify the sentiment tendency contained in the text, thereby greatly reducing the cost of data labelling. The method of constructing candidate labels for unlabelled data optimizes the performance of the semi-supervised learning model. The method solves the limitation of traditional co-training multi-view and domain knowledge. The method can predict new samples without adding the unlabelled data set to retrain the model.

[0244] Please refer to Figure 10The diagram illustrates an optional framework for a semi-supervised learning system. This system comprises two main subtasks, A10 and A20. A10 transforms unlabeled data into partially labeled data, constructing a candidate label set. A20 learns a model from these two different data formats, iteratively exchanging label information between the two models until convergence or the maximum number of iterations is reached. For details on the iterative exchange method, please refer to [link to details]. Figure 11 The diagram shows an optional training process.

[0245] In some embodiments of this application, please refer to the semi-supervised learning system framework diagram shown in Figure 10. The specific steps of the semi-supervised learning method provided in this application embodiment are as follows:

[0246] Step A1: Prepare training data.

[0247] This involves sampling any multi-label application scenario, such as images, audio, and text, to obtain training data. Relevant features are then extracted, and a small subset of data is manually labeled to obtain the training data: a small subset of labeled data. and a large amount of unlabeled data ,in For d-dimensional feature vectors, For example The mark, For containing q The label space for each category, and It can be done through Learn to obtain a classification model.

[0248] Step A2: Convert unlabeled data into partially labeled data.

[0249] This application involves assigning partial labels to unlabeled data, i.e., constructing a candidate label set to provide weak supervision information for the entire learning system, facilitating subsequent model derivation. The specific construction steps are as follows:

[0250] Step A21: Generate the kNN weight matrix: For each unlabeled sample In the labeled dataset Found Find the k nearest neighbors and label the data as Finally, a weight matrix can be obtained based on the relationship between the sample and its nearest neighbors. The calculation method for each element is as shown in formula (2-1):

[0251] Formula (2-1);

[0252] in, This is the bandwidth parameter of the kernel function, which defaults to 1 and can be adjusted according to the characteristics of the training data; for The 2-norm is the Euclidean distance between two eigenvectors.

[0253] For example, for 10 samples, L1 to L4 are labeled samples (L=4), and U1 to U6 are unlabeled samples (U=6); a weight matrix can be obtained. As shown in formula (2-2).

[0254] Formula (2-2);

[0255] For U1, if U1 has two nearest neighbors in L, including L3 and L4, then W11 and W21 corresponding to U1 are 0, and the elements determined by U1 include: .

[0256] Step A22: Generate the label confidence score.

[0257] Among them, KNN aggregation can be used to obtain each unlabeled sample. The confidence vector of the label As shown in formula (2-3).

[0258] Formula (2-3);

[0259] in, For example The mark, For labeled datasets middle The neighbor, exist The value is 1 if it is true, and 0 otherwise.

[0260] For example, based on the above example, the formula for calculating the label confidence of the k-th label in sample U1 is formula (2-4):

[0261] Formula (2-4);

[0262] It is a numerical value, which can be understood as when L3 and L4 contain Sometimes The calculation contributes to the result; otherwise, it is 0 with no contribution.

[0263] By calculating the label confidence scores for q preset categories for sample U1, a label confidence vector can be obtained. .

[0264] Step A23: Generate a candidate tag set.

[0265] wherein the above label confidence is normalized, each unlabeled sample candidate label can be obtained by formula (2-5):

[0266] formula (2-5);

[0267] final unlabeled dataset is converted into partial label dataset , which is used for subsequent model multi-class classifier derivation.

[0268] Step A3, model iterative update.

[0269] wherein the partial label model is learned from the partial label dataset, and the supervised model is learned from the labeled dataset, the supervision information contained in the two models is exchanged until the model converges or reaches the maximum iteration number, please refer to the training flowchart shown in Figure 11 . Taking the tth iteration as an example, it specifically includes the following steps:

[0270] Step A31, supervised model learning.

[0271] wherein, let be the classification model learned from the labeled dataset , wherein the sample x label prediction confidence can be obtained, and it satisfies . The supervised learning algorithm used can be freely set, that is , and the support vector machine is adopted in the present application. That is Figure 11 the supervised model B11 is obtained by the labeled data B10 in

[0272] Step A32, partial label model learning.

[0273] wherein, similar to the supervised model, let be the classification model learned from the partial label dataset , wherein the sample x label prediction confidence can be obtained, and it satisfies . The partial label learning algorithm used can be freely set, that is , and the IPAL is adopted in the present application. That is Figure 11 the partial label model B21 is obtained by the partial label data B20 in

[0274] Step A33, randomly sampling from the unlabeled data with replacement, updating the candidate label of the partial label data.

[0275] wherein, let From A subset of random samples is then used with the supervised model from step A31. predict The output is used to update the candidate labels of the biased label dataset.

[0276] For each sample ,set up For the corresponding samples in the partially labeled dataset, this application updates the candidate labels according to the following formula (2-6):

[0277] Formula (2-6);

[0278] in, It is a balancing parameter, controlling from and The relative proportion of inherited tag information.

[0279] The above steps are equivalent to Figure 11 In the middle, after the first sampling from the unlabeled data B30, the following was obtained. The data is then input into the supervised model B11 and the partially labeled model B21 to obtain "the confidence score of the sampled subset label of the supervised model" B12 and "the confidence score of the sampled subset label of the partially labeled model" B22, respectively. Combining B12 and B22, we can obtain "the confidence score of the updated sampled subset label, i.e. the candidate label of the partially labeled data" B40. We can use B40 to update the partially labeled data B20.

[0280] Step A34: Randomly sample again from the unlabeled data with replacement to expand the labeled dataset.

[0281] Among them, let From Another subset was randomly sampled from the data, and then the partial labeling model from step A32 was used. predict The labels are then added to the labeled dataset. Specifically, for each sample... , The prediction is labeled as formula (2-7):

[0282] Formula (2-7);

[0283] The above steps are equivalent to Figure 11 In the middle, after a second sampling from the unlabeled data B30, we obtain The data is then input into the partial labeling model B21 to obtain the "second sampling subset label" B50, which can be used to expand the labeled data B10.

[0284] after, and will be retrained based on the updated and into the t+1 iteration. The final learning model and is obtained after the iteration stops.

[0285] Step A4, label prediction.

[0286] where, for the unknown sample its final label is determined by the supervised model and the partial label model together, as formula (2-8):

[0287] Formula (2-8);

[0288] where, and are the empirical prediction accuracy of the original labeled data and .

[0289] The present application carries out experiments on some public multi-class data sets, and compares with four semi-supervised learning algorithms respectively: graph-based planetoid, optimization interval distribution-based ssodm, support vector machine-based S4VM, co-training-based cotrade, and a supervised learning algorithm: support vector machine-based LIBSVM. The experimental data only selects 0.5% labeled data, and the remaining 99.5% is unlabeled data, the experiment is repeated 10 times, the mean value is recorded, and the classification accuracy (mean±std) of the model on the public data set compared with the algorithm is shown in Table 1 below, • indicates that the performance of the model of the present application is significantly better than that of the comparative algorithm (paired t-test 0.05 significant level).

[0290] Table 1

[0291]

[0292] From the experimental results, the beneficial effects brought by the technical scheme of the present application can be summarized as follows:

[0293] (1) A new idea for using unlabeled data is proposed, which is no longer a single label as in previous semi-supervised learning methods, but a candidate label set, and the true label is contained in it. Solve the problem of label specification error and subsequent model performance decline caused by too few labeled samples and poor initial model performance. The comparative semi-supervised learning algorithm is single label specification, and from the experimental results, it can be seen that the idea adopted by the present application can bring significant performance improvement.

[0294] ​(2) Compared with the high requirement of data labeling of supervised learning, the application only needs a small amount of labeled data, can automatically mine supervision information from unlabeled data, greatly reduces the cost of manual labeling, and only uses 0.5% of the labeled data in the experiment, and fully utilizes a large amount of unlabeled data.

[0295] (3) Compared with the traditional semi-supervised learning method, the application is not simply binary classification, but is directly extended to multi-classification. The ssodm, S4VM and cotrade in the comparative algorithm are binary semi-supervised algorithms, and in order to adapt to multi-class data sets, "one-to-many" or "one-to-one" strategy is adopted for conversion, but the application directly provides an intuitive multi-classification method.

[0296] (4) It is verified through experiments that the prediction accuracy of the application is not inferior to the latest semi-supervised learning method, and especially excellent in the case of rare labeled data.

[0297] (5) Compared with the supervised learning algorithm LIBSVM, the performance of the application is much better in the case of rare labeled data.

[0298] In some embodiments of the application, the application can construct a candidate label set based on KNN and graph structure, but there are other strategies, such as label propagation. Moreover, the application is not limited to specific supervised learning methods and partial label learning methods, and other models can be replaced according to the actual application of memory time limit and accuracy requirement. For example, the supervised learning method can also use decision tree, logistic regression, etc., and the partial label learning method can be replaced by PL-kNN, PL-ECOC, etc.

[0299] The following continues to illustrate an exemplary structure of the implementation of the data classification device 555 provided by the embodiments of the application as a software module. In some embodiments of the application, as shown in FIG. 2, the software module stored in the data classification device 555 of the memory 550 includes:

[0300] The acquisition module 5551 is configured to acquire the data to be classified.

[0301] The classification module 5552 is configured to input the data to be classified into a first model and a second model of a multi-classification model respectively, and obtain a classification result of the data to be classified; wherein the first model is obtained by a first training set, and the first training set includes a plurality of first sample data carrying true classification labels; the second model is obtained by a second training set, and the second training set includes a plurality of second sample data carrying a partial label set, and the partial label set includes at least one candidate classification label.

[0302] In some embodiments of the present application, at least one of the first sample data is newly added, the real classification label carried by the newly added first sample data is obtained according to the second model, and / or at least one of the second sample data is updated, and the bias label set carried by the updated second sample data is obtained according to the first model.

[0303] In some embodiments of the present application, the classification module 5552 is further configured to: input the data to be classified into the first model to obtain a first classification result output by the first model, wherein the first classification result comprises a first confidence degree corresponding to each of the preset categories; input the data to be classified into the second model to obtain a second classification result output by the second model, wherein the second classification result comprises a second confidence degree corresponding to each of the preset categories; and determine the classification result of the data to be classified based on the first confidence degree and the second confidence degree corresponding to each of the preset categories.

[0304] In some embodiments of the present application, the classification module 5552 is further configured to: obtain a first precision parameter corresponding to the first classification result and a second precision parameter corresponding to the second classification result; the first precision parameter is used to represent the prediction precision of the first model; the second precision parameter is used to represent the prediction precision of the second model; determine a third confidence degree corresponding to each of the preset categories based on the first confidence degree and the second confidence degree corresponding to each of the preset categories, the first precision parameter and the second precision parameter; and take the preset category corresponding to the maximum third confidence degree as the classification result of the data to be classified.

[0305] In some embodiments of the present application, the data classification apparatus 555 further comprises a training module, which is configured to: obtain the first model through the first training set; obtain the second model through the second training set; perform a training set updating operation to obtain an updated training set in the case that a stop condition is not met, and iteratively train the multi-classification model by using the updated training set until the stop condition is reached, and output the trained multi-classification model; the training set updating operation comprises at least one of the following: updating the first training set by using the second model, and updating the second training set by using the first model.

[0306] The following continues to illustrate an exemplary structure of the data classification apparatus 655 provided by the embodiments of the present application, which is implemented as a software module. In some embodiments of the present application, as shown in FIG. 2, the software module stored in the multi-classification model training apparatus 655 of the memory 650 comprises:

[0307] The first model training module 6551 is configured to obtain the first model in the multi-classification model through the first training set, and the first training set comprises a plurality of first sample data carrying real classification labels.

[0308] The second model training module 6552 is configured to obtain a second model in the multi-classification model by using a second training set, and the second training set includes a plurality of second sample data carrying a bias label set, and the bias label set includes at least one candidate classification label.

[0309] The third model training module 6553 is configured to perform a training set updating operation in a case where the stop condition is not met, to obtain an updated training set, to iteratively train the multi-classification model by using the updated training set until the stop condition is reached, and to output the trained multi-classification model. The training set updating operation includes at least one of the following: updating the first training set by using the second model, and updating the second training set by using the first model.

[0310] In some embodiments of the present application, the multi-classification model training apparatus 655 further includes a training set construction module, which is configured to: construct the first training set and the third training set based on the original data set; the third training set includes a plurality of third sample data without classification labels; generate a bias label set corresponding to each third sample data according to each first sample data in the first training set; and add the bias label set corresponding to each third sample data to obtain the second training set.

[0311] In some embodiments of the present application, the training set construction module is further configured to: obtain a plurality of original data in the original data set; perform feature extraction on each original data to obtain original features corresponding to each original data, and construct an original training set; extract at least one first original feature and at least one second original feature from the original training set; construct the first training set including a plurality of first sample data based on the at least one first original feature and the real classification label corresponding to each original feature; and construct the third training set based on the at least one second original feature.

[0312] In some embodiments of the present application, the original data includes at least one of the following: voice data, text data, image data, and video data.

[0313] In some embodiments of the present application, the training set construction module is further configured to: generate a weight matrix according to the similarity between each third sample data and each first sample data; generate a category confidence vector corresponding to each third sample data based on the weight matrix; the category confidence vector includes a fourth confidence corresponding to each preset category; and generate a bias label set corresponding to each third sample data according to the category confidence vector corresponding to each third sample data.

[0314] In some embodiments of the present application, the training set construction module is further configured to: generate a weight vector corresponding to each third sample data according to the similarity between each third sample data and each first sample data; and generate a weight matrix according to the weight vector corresponding to each third sample data.

[0315] In some embodiments of the application, the training set construction module is further configured to: in each first sample data, determine a set of neighbor samples of the third sample data; the set of neighbor samples includes at least one neighbor sample data; the neighbor sample data is similar to the third sample data; when the first sample data is in the set of neighbor samples, generate the weight element corresponding to the first sample data using a first weight generation rule; when the first sample data is not in the set of neighbor samples, generate the weight element corresponding to the first sample data using a second weight generation rule; and generate the weight vector corresponding to the third sample data based on the weight element corresponding to each sample data.

[0316] In some embodiments of the application, the training set construction module is further configured to: determine, according to each first sample data, a fourth confidence of the third sample data in each preset category; and determine, according to the fourth confidence of the third sample data in each preset category, a category confidence vector corresponding to the third sample data.

[0317] In some embodiments of the application, the training set construction module is further configured to: normalize the category confidence vector corresponding to the third sample data to obtain a normalized category confidence vector; the normalized category confidence vector includes a fifth confidence of each preset category; and determine, according to the fifth confidence of each preset category, a set of bias labels corresponding to the third sample data.

[0318] In some embodiments of the application, the third model training module 6553 is further configured to: obtain at least one to-be-updated sample data in the third training set; input each to-be-updated sample data into the first model and the second model to obtain a set of to-be-updated bias labels corresponding to each to-be-updated sample data; and update the second training set according to the set of to-be-updated bias labels corresponding to each to-be-updated sample data.

[0319] In some embodiments of the application, the third model training module 6553 is further configured to: input the to-be-updated sample data into the first model to obtain a third classification result output by the first model; input the to-be-updated sample data into the second model to obtain a fourth classification result output by the second model; and determine a set of to-be-updated bias labels corresponding to the to-be-updated sample data according to the third classification result and the fourth classification result.

[0320] In some embodiments of the present application, the third classification result includes a sixth confidence degree corresponding to each preset category in the plurality of preset categories; the fourth classification result includes a seventh confidence degree corresponding to each preset category; the third model training module 6553 is further configured to: based on a preset balance parameter, perform weighted summation on the sixth confidence degree and the seventh confidence degree corresponding to each preset category to obtain an eighth confidence degree corresponding to each preset category; and determine a to-be-updated bias label set corresponding to the to-be-updated sample data according to the eighth confidence degree of each preset category.

[0321] In some embodiments of the present application, the third model training module 6553 is further configured to: query to-be-updated second sample data corresponding to the to-be-updated sample data in the second training set; and update the bias label set carried by the to-be-updated second sample data by using the to-be-updated bias label set.

[0322] In some embodiments of the present application, the third model training module 6553 is further configured to: obtain at least one to-be-labeled sample data in the third training set; input each to-be-labeled sample data into the second model to obtain a real classification label corresponding to each to-be-labeled sample data; and update the first training set according to the real classification label corresponding to each to-be-labeled sample data.

[0323] In some embodiments of the present application, the third model training module 6553 is further configured to: input the to-be-labeled sample data into the second model to obtain a fifth classification result output by the second model; and determine a real classification label corresponding to the to-be-labeled sample data according to the fifth classification result.

[0324] In some embodiments of the present application, the fifth classification result includes a ninth confidence degree corresponding to each preset category in the plurality of preset categories; and the third model training module 6553 is further configured to: take a preset category corresponding to a maximum ninth confidence degree as the real classification label corresponding to the to-be-labeled sample data.

[0325] In some embodiments of the present application, the third model training module 6553 is further configured to: add a corresponding real classification label to the to-be-labeled sample data; and add the to-be-labeled sample data with the added real classification label to the first training set as new first sample data.

[0326] In some embodiments of the present application, the stop condition includes at least one of the following: the multi-classification model tends to converge; and a maximum iteration number is reached.

[0327] The embodiment of the present disclosure provides a computer program product or a computer program, the computer program product or the computer program comprises computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data classification method or the multi-classification model training method provided in the embodiment of the present disclosure.

[0328] The embodiment of the present disclosure provides a computer readable storage medium storing executable instructions, wherein the executable instructions are stored, when the executable instructions are executed by a processor, the processor will execute the data classification method or the multi-classification model training method provided by the embodiment of the present disclosure, for example, as Figures 3 to 8 The method shown.

[0329] In some embodiments of the present application, the computer readable storage medium can be FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM memory, etc. It can also be various devices including one or any combination of the above memories.

[0330] In some embodiments of the present application, the executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or as modules, components, subroutines or other units suitable for use in a computing environment.

[0331] As an example, the executable instructions can but not necessarily correspond to files in a file system, can be stored in part of a file storing other programs or data, for example, stored in one or more scripts in a hyper text markup language (HTML, Hyper Text Markup Language) document, stored in a single file dedicated to the program in question, or stored in multiple cooperative files (for example, files storing one or more modules, subroutines or code portions).

[0332] As an example, the executable instructions can be deployed to execute on one computing device, or on multiple computing devices located in one place, or on multiple computing devices distributed in multiple places and interconnected through a communication network.

[0333] In summary, through the embodiments of the present application, the following technical effects can be achieved:

[0334] (1) The embodiment of the present application utilizes the first model obtained based on a plurality of first sample data carrying real classification labels and the second model obtained based on a plurality of second sample data carrying bias label sets to classify the data to be classified. Since the multi-classification model integrates the classification results of the first model and the classification results of the second model, the classification result output by the multi-classification model can be closer to the actual classification of the data to be classified. In this way, the accuracy of data classification can be improved.

[0335] (2) The embodiment of the present application obtains the first precision parameter corresponding to the first model and the second precision parameter corresponding to the second model, and weights the first classification result output by the first model by the first precision parameter and weights the second classification result by the second precision parameter. The final classification result not only includes the classification results of the two classification models, but also combines the classification accuracy of each classification model. In this way, the obtained classification result can be more accurate.

[0336] (3) In the model training process, the embodiment of the present application respectively utilizes the first model to update the second training set used to obtain the second model, and utilizes the second model to update the first training set used to obtain the first model. In the process of completing the training information interaction between the first model and the second model, the classification accuracy of the first model and the second model can be continuously improved, and thus the classification accuracy of the overall multi-classification model can be improved. Meanwhile, in the process of updating the first training set by the second model in the embodiment of the present application, the first sample data is mined, the workload of manual labeling is greatly reduced, and the overall efficiency of data classification is improved. Meanwhile, the embodiment of the present application can accelerate the convergence speed of the multi-classification model and improve the model training efficiency by the way of mutual updating of the training set of the first model and the second model.

[0337] (4) In the model training process, the embodiment of the present application respectively utilizes the first model to update the second training set used to obtain the second model, and utilizes the second model to update the first training set used to obtain the first model. In the process of completing the training information interaction between the first model and the second model, the classification accuracy of the first model and the second model can be continuously improved, and thus the classification accuracy of the overall multi-classification model can be improved. Meanwhile, in the process of updating the first training set by the second model in the embodiment of the present application, the first sample data is mined, the workload of manual labeling is greatly reduced, and the overall efficiency of data classification is improved. Meanwhile, the embodiment of the present application can accelerate the convergence speed of the multi-classification model and improve the model training efficiency by the way of mutual updating of the training set of the first model and the second model.

[0338] (5) The embodiment of the present application generates the corresponding bias label set for each third sample data by acquiring the near neighbor sample set corresponding to each third sample data in the first training set. In this way, a large amount of second sample data carrying supervision information can be automatically generated under the premise of very few labeled data to obtain the second training set, and the second model under the semi-supervised learning method can be obtained through the second training set, which provides a data basis for the model precision of the subsequent overall multi-classification model. At the same time, since the second model is trained by using the second sample data carrying the bias label set including multiple candidate classification labels, compared with the semi-supervised learning method using only a single candidate label in the related art, the problem of label assignment error caused by too few labeled samples and poor initial model performance and the subsequent decline in model performance is solved, and the second model obtained by the embodiment of the present application can solve the multi-classification problem. Compared with the semi-supervised model in the related art which can only solve the binary classification problem, the performance of the second model obtained by the present application is more superior.

[0339] (6) The embodiment of the present application updates the multiple candidate classification labels carried by the second sample data by the first model, so that the updated second sample data can be more close to the actual classification result. In this way, the second model with higher prediction accuracy can be obtained through the updated second training set, which provides a basis for improving the overall model precision of the multi-classification model.

[0340] (7) The embodiment of the present application adds new first sample data carrying real classification labels to the first training set by the second model. On the basis of the original small amount of first sample data, the updated first training set can be more abundant due to the increase in the number of first sample data. In this way, the first model with higher prediction accuracy can be obtained through the updated first training set, which provides a basis for improving the overall model precision of the multi-classification model.

[0341] The above is only an embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image data classification method, characterized in that, include: Obtain the image to be classified; The image to be classified is input into the first and second models of the multi-classification model respectively to obtain the classification result of the image to be classified; The first model is obtained through a first training set, which includes multiple first sample data carrying true classification labels. The second model is obtained through a second training set, which includes multiple second sample data carrying a set of partial labels. The first sample data and the second sample data are image data, and the set of partial labels includes at least one candidate classification label. The second training set is obtained by: generating a set of partial labels corresponding to each third sample data based on each first sample data in the first training set; adding the corresponding set of partial labels to each third sample data to obtain the second training set, wherein the third sample data has no classification label. During the training of the multi-classification model, the first and second training sets are updated in the following manner: The first training set is updated using the second model, by adding at least one first sample data carrying the true classification label to the first training set; Furthermore, the second sample data in the second training set is updated using the first model and the second model to obtain at least one updated second sample data carrying a set of biased labels.

2. The method according to claim 1, characterized in that, The step of inputting the image to be classified into the first and second models of a multi-classification model respectively to obtain the classification result of the image to be classified includes: The image to be classified is input into the first model to obtain a first classification result output by the first model; the first classification result includes a first confidence level corresponding to each of the multiple preset categories; The image to be classified is input into the second model to obtain a second classification result output by the second model; the second classification result includes a second confidence level corresponding to each of the plurality of preset categories; The classification result of the image to be classified is determined based on the first confidence level and the second confidence level corresponding to each preset category.

3. The method according to claim 2, characterized in that, The step of determining the classification result of the image to be classified based on the first confidence score and the second confidence score corresponding to each preset category includes: Obtain a first precision parameter corresponding to the first classification result and a second precision parameter corresponding to the second classification result; the first precision parameter is used to characterize the prediction accuracy of the first model; the second precision parameter is used to characterize the prediction accuracy of the second model. Based on the first confidence level and the second confidence level, the first precision parameter and the second precision parameter corresponding to each preset category, a third confidence level corresponding to each preset category is determined; The preset category corresponding to the maximum third confidence level is used as the classification result of the image to be classified.

4. A multi-classification model training method, characterized in that, include: The first model in the multi-class classification model is obtained through the first training set, which includes multiple first sample data carrying true classification labels. The second model in the multi-class classification model is obtained through a second training set, which includes multiple second sample data carrying a set of partial labels, the set of partial labels including at least one candidate classification label; wherein, the first sample data and the second sample data are image data, and the second training set is obtained by: generating a set of partial labels corresponding to each third sample data according to each first sample data in the first training set; adding the corresponding set of partial labels to each third sample data to obtain the second training set, wherein the third sample data has no classification label; If the stopping condition is not met, a training set update operation is performed to obtain an updated training set. The multi-classification model is then iteratively trained using the updated training set until the stopping condition is met, and the trained multi-classification model is output. The training set update operation includes at least one of the following: updating the first training set using the second model, adding at least one first sample data carrying the true classification label to the first training set, and updating the second sample data in the second training set using the first model and the second model to obtain at least one updated second sample data carrying a set of biased labels.

5. The method according to claim 4, characterized in that, The method further includes: Based on the original dataset, a first training set and a third training set are constructed; the third training set includes multiple third sample data without classification labels; Based on each first sample data in the first training set, generate a set of partial labels corresponding to each third sample data. Add a corresponding set of partial labels to each of the third sample data to obtain the second training set.

6. The method according to claim 5, characterized in that, The step of generating a partial label set corresponding to each third sample data based on each first sample data in the first training set includes: A weight matrix is ​​generated based on the similarity between each third sample data and each first sample data. Based on the weight matrix, a category confidence vector is generated for each of the third sample data; the category confidence vector includes a fourth confidence value for each preset category; Generate a set of partial labels corresponding to each of the third sample data based on the category confidence vector corresponding to each of the third sample data.

7. The method according to claim 6, characterized in that, The step of generating a weight matrix based on the similarity between each of the third sample data and each of the first sample data includes: Based on the similarity between each third sample data and each first sample data, a weight vector is generated corresponding to each third sample data. The weight matrix is ​​generated based on the weight vector of each of the third sample data pairs.

8. The method according to claim 7, characterized in that, The step of generating a weight vector corresponding to each third sample data based on the similarity between each third sample data and each first sample data includes: In each of the first sample data, a set of nearest neighbor samples of the third sample data is determined; the set of nearest neighbor samples includes at least one nearest neighbor sample data; the nearest neighbor sample data is the first sample data that is similar to the third sample data. When the first sample data is in the nearest neighbor sample set, the weight elements corresponding to the first sample data are generated using the first weight generation rule; If the first sample data is not in the nearest neighbor sample set, the weight elements corresponding to the first sample data are generated using the second weight generation rule; Based on the weight elements corresponding to each of the sample data, a weight vector corresponding to the third sample data is generated.

9. The method according to claim 6, characterized in that, The step of generating a category confidence vector for each of the third sample data based on the weight matrix includes: Based on each of the first sample data, determine the fourth confidence level of the third sample data in each of the preset categories; Based on the fourth confidence level of the third sample data in each preset category, determine the category confidence vector corresponding to the third sample data.

10. The method according to claim 5, characterized in that, The step of updating the second sample data in the second training set using the first model and the second model to obtain at least one updated set of second sample data carrying a biased label includes: At least one sample data to be updated is obtained from the third training set; Each of the sample data to be updated is input into the first model and the second model to obtain the set of partial labels to be updated corresponding to each sample data to be updated. The second training set is updated according to the set of partial labels to be updated corresponding to each of the sample data to be updated.

11. The method according to claim 5, characterized in that, The step of updating the first training set using the second model, by adding at least one first sample data carrying the true classification label to the first training set, includes: At least one unlabeled sample data is obtained from the third training set; Each of the unlabeled sample data is input into the second model to obtain the true classification label corresponding to each of the unlabeled sample data. The first training set is updated based on the true classification label corresponding to each of the unlabeled sample data.

12. A data classification device, characterized in that, include: The acquisition module is used to acquire the image to be classified. A classification module is used to input the image to be classified into a first model and a second model of a multi-classification model to obtain the classification result of the image to be classified; wherein, the first model is obtained through a first training set, the first training set including multiple first sample data carrying true classification labels; the second model is obtained through a second training set, the second training set including multiple second sample data carrying a set of partial labels, the first sample data and the second sample data are image data, and the set of partial labels includes at least one candidate classification label; wherein, the second training set is obtained by: generating a set of partial labels corresponding to each third sample data according to each first sample data in the first training set; adding the corresponding set of partial labels to each third sample data to obtain the second training set, wherein the third sample data has no classification label; During the training of the multi-classification model, the first training set and the second training set are updated in the following ways: the first training set is updated by the second model, adding at least one first sample data carrying the true classification label to the first training set; and the second sample data in the second training set is updated by the first model and the second model, to obtain at least one updated second sample data carrying a set of biased labels.

13. The apparatus according to claim 12, characterized in that, The classification module is also used for: The image to be classified is input into the first model to obtain a first classification result output by the first model; the first classification result includes a first confidence level corresponding to each of the multiple preset categories; The image to be classified is input into the second model to obtain the second classification result output by the second model. The second classification result includes a second confidence level corresponding to each of the plurality of preset categories; The classification result of the image to be classified is determined based on the first confidence level and the second confidence level corresponding to each preset category.

14. A multi-classification model training device, characterized in that, include: The first model training module is used to obtain the first model in the multi-class model through the first training set, which includes multiple first sample data carrying true classification labels. The second model training module is used to obtain a second model in the multi-classification model through a second training set. The second training set includes multiple second sample data carrying a set of partial labels, and the set of partial labels includes at least one candidate classification label. The first sample data and the second sample data are image data. The second training set is obtained by: generating a set of partial labels corresponding to each third sample data based on each first sample data in the first training set; adding the corresponding set of partial labels to each third sample data to obtain the second training set, wherein the third sample data has no classification label. The third model training module is used to perform a training set update operation if the stopping condition is not met, to obtain an updated training set, and to iteratively train the multi-classification model using the updated training set until the stopping condition is met, and output the trained multi-classification model; the training set update operation includes at least one of the following: updating the first training set through the second model, adding at least one first sample data carrying the true classification label to the first training set, and updating the second sample data in the second training set through the first model and the second model to obtain at least one updated second sample data carrying a set of biased labels.

15. A computer device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method of any one of claims 1 to 3, or implements the method of any one of claims 4 to 11.

16. A computer-readable storage medium, characterized in that, It stores executable instructions for use by a processor to implement the method of any one of claims 1 to 3, or the method of any one of claims 4 to 11.

17. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method according to any one of claims 1 to 3, or the method according to any one of claims 4 to 11.

Images