A mining method and device for data regular expressions

Abstract

The invention provides a data regular expression mining method. The method includes: acquiring and storing data and storing it in a dictionary tree structure; upgrading nodes according to regular expression rules; The number of nodes is merged separately; the interference branch is identified, and the branch is deleted; the rule tree is converted into a string format and output. By storing the acquired data in a dictionary tree structure, it is possible to mine massive data, by upgrading data nodes, merging branches, deleting interfering branches, and finally converting the generated rule tree into a string format for input. The invention realizes the mining of the regular expressions of massive data containing wrong data, the rule tree can satisfy the mining of wrong data, and can be used to check the data and find out the wrong data. In addition, the present invention also provides a data regular expression mining device.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a data regular expression mining method and device. Background technique [0002] Data mining refers to the process of extracting information that people do not know but is valuable to users from a large amount of incomplete, vague, and erroneous data. The data mining process usually includes preprocessing data, implementing data mining algorithms, and displaying mining results. The early data mining process was implemented in a serial manner on a stand-alone node. In a stand-alone node data mining system, the amount of data that can be mined and the load of the algorithm depend on the performance of a single execution node. Since the current data mining systems need to deal with massive data, this method of serial processing on a single node can only support a small amount of data, and its performance is low. Later, with the development of data mining technology, the current min...

AI Technical Summary

Problems solved by technology

There are many methods and techniques for applying regular expressions, but there are few ways to generate a more effective regular expression. For example, Sergei Savchenko proposed a method based on The regular expression mining method of intelligent finite automata, but this method also has great limitations, for example, the method has distribution requirements and the size of the data set can only be between 30-50

[0004] At present, in the field of data processing, there is no mining method that can mine the essential structure of data and form a regular expression for massive data containing error data.

Images