Machine learning-based data classification method and device

Abstract

The present invention provides a machine learning-based data classification method. The method comprises the following steps of S11, based on learning data, determining a first feature word group corresponding to each data; S12, according to feature words, classifying the learning data; S13, judging whether the classification of the learning data is correct or not, and if yes, jumping to the step S15; if not, adjusting the first feature word group, and jumping to the step S12; S15, establishing a data classification model based on the first feature word group. The machine learning-based data classification device comprises a first feature word group determination module, a first data classification module, a judgment and classification module, a second feature word group determination module and a modeling module. According to the technical scheme of the invention, the content of a file is subjected to word-cutting treatment, and the weight of a word is calculated based on the TFIDF algorithm. After that, the similarity of the file is calculated. Similar files are clustered and feature words are extracted. Feature words are different from key words, while feature words are more representative and more suitable to be served as the sensitive information. Therefore, the feature words of one cluster can be distinguished from the feature words of other clusters.

Description

technical field [0001] The invention relates to the field of data classification, in particular to a data classification method and equipment based on machine learning. Background technique [0002] As the importance of critical data in ensuring enterprise operations continues to rise, managers are increasingly concerned about data security. At the same time, leaks of sensitive documents occur frequently, but it is obviously difficult to identify and classify sensitive documents only by humans. For a batch of files, it is difficult to accurately distinguish which files belong to which category only by the file name or file type. For the extraction of sensitive data, it is obviously very difficult to filter and screen and extract sensitive data only by manual methods. [0003] Internal personnel themselves lack the experience, motivation, and technical support to complete the data combing work. The existing sensitive data discovery technology architecture generally require...

AI Technical Summary

Problems solved by technology

[0005] (1) The clustering results are unsatisfactory. When the clustering results are obtained, the results are often directly based on the separated results, and more flexible adjustments cannot be made.

In fact, some words have polysemy and have different meanings in different documents, including idioms, proverbs, etc. These words are likely to be separated during the word segmentation process, which leads to the original very representative words becoming different. It has become an ordinary word, which will seriously affect the clustering effect

[0006] (2) The keywords are too short, and they may also appear in other types of documents, so it is necessary to provide enough keywords to be the characteristics of this type of document, and the process of selecting keywords is not very easy , if the number is too large, it will take a considerable amount of time

If all are selected to save time, a considerable part of the vocabulary will be redundant, which will affect the matching results and efficiency to a certain extent

If only a part of the words are selected, it is necessary to manually compare and screen the articles, which is obviously not advisable

[0007] (3) The process is too complicated, the operation is not coherent, and it is difficult to get started. It is likely to miss a certain link during the entire operation process, or waste unnecessary time

Images