Data classification method and device, computer device and storage medium

Abstract

The invention provides a data classification method and device, a computer device and a storage medium. The method comprises the steps of obtaining a to-be-labeled data set; labeling the data set through a labeling function to obtain an initial label of the data set; calculating a pair correlation of the identification function according to the initial label, and constructing a generation model ofthe marking function according to the pair correlation; estimating a probability label of the data set according to the generation model; training a discrimination model according to the probabilitylabel to obtain a trained discrimination model; and inputting to-be-classified data into the trained discrimination model to obtain the category of the to-be-classified data. According to the method,the marking efficiency and accuracy of the training data are improved, the training data can be utilized to quickly train the discrimination model, and the discrimination model is utilized to realizethe quick and accurate data classification.

Description

technical field

[0001] The invention relates to the technical field of machine learning, in particular to a data classification method, device, computer device and computer storage medium. Background technique

[0002] With the rapid development of artificial intelligence, machine learning technology (especially deep learning technology) has been applied in various industries. At this point, training data labeling has gradually become the biggest bottleneck in the widespread deployment of machine learning systems.

[0003] The existing traditional manual labeling methods are time-consuming, labor-intensive and costly, and existing data enhancement methods such as semi-supervised learning, active learning, and transfer learning cannot quickly generate large-scale training data.

[0004] How to formulate a suitable solution, reduce the workload of manually labeling training data, and improve the efficiency of labeling training data is a technical problem that relevant technic...

Examples

Embodiment 1

[0067] figure 1 It is a flow chart of the data classification method provided by Embodiment 1 of the present invention. The data classification method is applied to a computer device.

[0068] The data classification method of the present invention is applied to a machine learning system for generating training data, using the training data to train a discriminant model of the machine learning system, and using the trained discriminant model to classify the data to be classified. The data classification method can quickly generate the training data required by the discriminant model of the machine learning system, solve the technical problems that manual labeling training data is difficult to obtain, the labeling time is long, and the accuracy rate cannot be guaranteed, and the work of manual labeling training data can be reduced Quantity, improve the labeling efficiency and accuracy of the training data, use the training data to quickly train the discriminant model, and use ...

Embodiment 2

[0125] figure 2 It is a structural diagram of the data classification device provided by Embodiment 2 of the present invention. The data classification device 20 is applied to a machine learning system for generating training data, using the training data to train a discriminant model of the machine learning system, and using the trained discriminant model to classify the data to be classified. The data classification device 20 can quickly generate the training data required by the discriminant model of the machine learning system, solve the technical problems that the manual labeling training data is difficult to obtain, the labeling time is long, and the accuracy rate cannot be guaranteed, and the manual labeling training data is reduced. workload, improve the labeling efficiency and accuracy of the training data, use the training data to quickly train the discriminant model, and use the discriminant model to achieve fast and accurate data classification.

[0126] like f...

Embodiment 3

[0182] This embodiment provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above embodiment of the data classification method are implemented, for example figure 1 Steps 101-106 shown:

[0183] Step 101, obtain the data set {x to be labeled i |i=1,2,...,m};

[0184] Step 102, by labeling the function λ j , j=1, 2,..., n to mark the data set, to obtain the initial label Λ of the data set i，j =λ j (x i ), i=1, 2, ..., m, j = 1, 2, ..., n;

[0185] Step 103, calculating the pairwise correlation of the labeling function according to the initial label, and constructing a generative model of the labeling function according to the pairwise correlation;

[0186] Step 104, estimating the probability label of the data set according to the generation model;

[0187] Step 105, training the discriminant model of the machine learning system according to the probability label to obtain the tr...

technical field

[0001] The invention relates to the technical field of machine learning, in particular to a data classification method, device, computer device and computer storage medium. Background technique

[0002] With the rapid development of artificial intelligence, machine learning technology (especially deep learning technology) has been applied in various industries. At this point, training data labeling has gradually become the biggest bottleneck in the widespread deployment of machine learning systems.

[0003] The existing traditional manual labeling methods are time-consuming, labor-intensive and costly, and existing data enhancement methods such as semi-supervised learning, active learning, and transfer learning cannot quickly generate large-scale training data.

[0004] How to formulate a suitable solution, reduce the workload of manually labeling training data, and improve the efficiency of labeling training data is a technical problem that relevant technic...

Examples

Embodiment 1

[0067] figure 1 It is a flow chart of the data classification method provided by Embodiment 1 of the present invention. The data classification method is applied to a computer device.

[0068] The data classification method of the present invention is applied to a machine learning system for generating training data, using the training data to train a discriminant model of the machine learning system, and using the trained discriminant model to classify the data to be classified. The data classification method can quickly generate the training data required by the discriminant model of the machine learning system, solve the technical problems that manual labeling training data is difficult to obtain, the labeling time is long, and the accuracy rate cannot be guaranteed, and the work of manual labeling training data can be reduced Quantity, improve the labeling efficiency and accuracy of the training data, use the training data to quickly train the discriminant model, and use ...

Embodiment 2

[0125] figure 2 It is a structural diagram of the data classification device provided by Embodiment 2 of the present invention. The data classification device 20 is applied to a machine learning system for generating training data, using the training data to train a discriminant model of the machine learning system, and using the trained discriminant model to classify the data to be classified. The data classification device 20 can quickly generate the training data required by the discriminant model of the machine learning system, solve the technical problems that the manual labeling training data is difficult to obtain, the labeling time is long, and the accuracy rate cannot be guaranteed, and the manual labeling training data is reduced. workload, improve the labeling efficiency and accuracy of the training data, use the training data to quickly train the discriminant model, and use the discriminant model to achieve fast and accurate data classification.

[0126] like f...

Embodiment 3

[0182] This embodiment provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above embodiment of the data classification method are implemented, for example figure 1 Steps 101-106 shown:

[0183] Step 101, obtain the data set {x to be labeled i |i=1,2,...,m};

[0184] Step 102, by labeling the function λ j , j=1, 2,..., n to mark the data set, to obtain the initial label Λ of the data set i，j =λ j (x i ), i=1, 2, ..., m, j = 1, 2, ..., n;

[0185] Step 103, calculating the pairwise correlation of the labeling function according to the initial label, and constructing a generative model of the labeling function according to the pairwise correlation;

[0186] Step 104, estimating the probability label of the data set according to the generation model;

[0187] Step 105, training the discriminant model of the machine learning system according to the probability label to obtain the tr...

Images