Data validity check system

Abstract

The invention relates to a data validity check system. The system comprises one or more clients, one or more uploading servers, a distributed data validity check platform, a management server and oneor more downloading servers. The management server carries out task management on check resources in the distributed data validity check platform, manages the uploading servers and the downloading servers and provides service for the clients. According to the data validity check system, the parallel processing can be carried out on batch data, and the automatic, high-secure and high-parallel datavalidity check can be carried out without depending on the capability of a user.

Description

【Technical field】 [0001] The invention belongs to the field of data processing, in particular to a data validity checking system. 【Background technique】 [0002] Data validity inspection is based on the judgment method to judge whether the data to be inspected is valid. The application range of validity verification is very wide. For example: validity verification is a common step in data cleaning, and it is the first important step after data entry. It is the basis of preprocessing before the data is further processed. In the specific inspection process, if the user enters gender information, it can be judged whether the value of the gender information is "male" or "female". If not, then such information is obviously invalid, and invalid data cannot be used for further analysis. , nor can it be used as the basis for further work. [0003] The data validity verification tools in the prior art are very simple, usually based on excel or a simple form for verification, and sl...

AI Technical Summary

Problems solved by technology

In the specific inspection process, if the user enters gender information, it can be judged whether the value of the gender information is "male" or "female". If not, then such information is obviously invalid, and invalid data cannot be used for further analysis. , nor can it be used as the basis for further work

[0003] The data validity verification tools in the prior art are very simple, usually based on excel or simple forms for verification, and slightly more complicated verification methods require users to write verification programs by themselves

However, with the advent of the big data era, the scale of verification data is massive, and the verification rules required by users cannot be realized through simple excel, and the cost of writing programs for each rule is unimaginable. Based on the above-mentioned problems, now There is an urgent need for a new data validity verification method and system that can process batch data in parallel, independent of the user's own capabilities, and can perform automated, high-security, and high-parallel data validity verification

Images