Government affair data quality assessment and abnormal data restoration technology based on big data technology

Abstract

The invention discloses a government affair data quality assessment and abnormal data restoration technology based on big data technology in the technical field of data analysis, comprising the following steps: firstly establishing a database, then carrying out data quality assessment, and finally carrying out data quality restoration. Null value, value domain, specification, logic, citation and repeated data inspection are carried out on data fields. The data quality is comprehensively evaluated from six dimensions of data integrity, relevance, uniqueness, accuracy, consistency and normalization. A data quality evaluation report is generated, and a user carries out manual repair or rule repair or deep learning repair on the data to help the government to break through internal data barriers, disable data assets, improve data values, provide uniform intelligent data services to the outside and further deeply dig and release value bonus of big data.

Description

technical field [0001] The invention relates to the technical field of data analysis, in particular to a technology for evaluating the quality of government affairs data and repairing abnormal data based on big data technology. Background technique [0002] The project is based on the quality management method of PDCA (Plan, Do, Check, Act, American quality management expert Dr. Shewhart, later adopted and popularized by Deming), DQAF (Data Quality Assessment Framework, IMF and the World Bank released international data Quality Assessment Framework) data quality assessment model, DAMA (International Data Management Association) data management functional framework and deep learning-based abnormal data repair technology, establish a complete big data science governance system and standards, ensure data quality, and improve government services Efficiency, ensuring the high efficiency and smooth flow of the data infrastructure highway, laying a solid foundation for the construc...

AI Technical Summary

Problems solved by technology

[0003] At present, all government departments have related database systems, which are cut and managed, resulting in unsmooth government information and messy data in government databases. There are various problems in a large amount of data that are easy to find and find, which can easily lead to data loss and inaccuracies. Accurate, such as missing or incorrectly filling in the ID card number in the population database, incomplete or wrong information in the legal person database, etc., using this big data analysis and evaluation method, the entire government information platform, from data integrity, relevance, Comprehensive evaluation of data quality from six dimensions of uniqueness, accuracy, consistency, and normativeness

Images