Data repetition frequency monitoring method based on big data

Abstract

The invention discloses a data repetition frequency monitoring method based on big data, and the method comprises the following steps: numbering to-be-monitored data according to the types of the data, and integrating the data with the same number; and determining a keyword of the data to be monitored, and detecting the similarity between the data to be monitored and the same type of data by adopting a field matching method. According to the data repetition frequency monitoring method based on big data, the data in comparison is the same as the data to be monitored in type and adopts a corresponding data recording mode; the comparison data volume can be greatly reduced; the data processing process is accelerated, the requirements of data comparison on equipment are reduced, people can conveniently know the key content of data, people can conveniently analyze the data, people can conveniently and quickly judge the change condition of the data, the workload of subsequent manual data analysis of people is reduced, people can conveniently sort out the data, and a better use prospect is brought.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a method for monitoring data repetition frequency based on big data. Background technique [0002] With the development of society, the industry and the big data industry continue to develop. In order to facilitate people to better manage their own data through big data, people have invented some data processing methods, including data repetition frequency monitoring methods based on big data. With the development of science and technology, people have higher and higher requirements for data repetition frequency monitoring methods based on big data, resulting in the existing data repetition frequency monitoring methods based on big data cannot meet people's requirements; [0003] The existing data repetition frequency monitoring method based on big data has certain disadvantages in use. The existing data repetition frequency monitoring method based on big data usually compares the ...

AI Technical Summary

Problems solved by technology

[0002] With the development of society, the industry and the big data industry continue to develop. In order to facilitate people to better manage their own data through big data, people have invented some data processing methods, including data repetition frequency monitoring methods based on big data. With the development of science and technology, people have higher and higher requirements for data repetition frequency monitoring methods based on big data, resulting in the existing data repetition frequency monitoring methods based on big data cannot meet people's requirements;

[0003] The existing data repetition frequency monitoring method based on big data has certain disadvantages in use. The existing data repetition frequency monitoring method based on big data usually compares the data to be monitored with all the data. The amount is large, and the requirements for equipment are high, and the existing data repetition frequency monitoring method based on big data only detects the overall repetition rate of the data, and then deletes the redundant data without further processing of the data, which is inconvenient for people to organize the data. Does not meet people's requirements, so we propose a data repetition frequency monitoring method based on big data

Images