Data quality-based data management system

A data quality and data technology, applied in the field of data governance, can solve problems such as lack of data governance, achieve the effect of improving consistency and standardization, improving usability and operational efficiency, and improving data governance capabilities

Pending Publication Date: 2018-03-02

SHANGHAI DEV CENT OF COMP SOFTWARE TECH +1

5 Cites 9 Cited by

AI-Extracted Technical Summary

Problems solved by technology

[0003] However, in the field of data governance, there has not been a complete plan on how to evaluate th...

Method used

Finally build data quality problem knowledge base based on above-mentioned three st...

Abstract

The invention discloses a data quality-based data management system. According to the data management system, metadata is collected, and a metadatabase of a specified system is formed by configuring an underlying data source and a suspension point; the data in the metadatabase is subjected to information isolated land screening, namely, a primary and foreign key association relationship collectedby the metadatabase is extracted, information isolated land data forming a data stream connection without assistance of primary and foreign key association is displayed in a list, a user is prompted to perform modification and perfection, and scoring is performed according to a problem data quantity ratio; and field names, field types and field lengths in the metadata are compared with a standarddata dictionary, any piece of unconformable metadata is extracted and displayed in a front-end UI, data standardization condition assessment is realized, and the scoring is performed according to theproblem data ratio.

Application Domain

Special data processing applicationsDatabase design/maintainance

Technology Topic

Management systemStandardization +7

Image

Examples

Experimental program(1)

Example Embodiment

[0018] like figure 1 As shown in the figure, the system collects metadata, and forms the metadata database of the specified system by configuring the underlying data sources and suspension points. Secondly, the data in the metadata database is screened for information islands, that is, by extracting the primary and foreign key associations collected by the metadata database, on the front-end page, the information island data that is not connected to the data flow through the primary and foreign key associations will be listed in the list. It is displayed in the display, prompting the user to modify and improve, and at the same time, it is scored according to the proportion of the number of question data.

[0019] Next, compare the field name, field type, field length, etc. in the metadata with the data standard dictionary, extract any inconsistent metadata and display it on the front-end UI interface to realize the evaluation of data standardization, and at the same time according to the problem data. The ratio is scored.

[0020] The next step is to evaluate the data content. First, the evaluation indicators must be customized, including several indicators such as completeness, consistency, accuracy, uniqueness, effectiveness, timeliness and security. The system provides a template for the formulation of inspection indicators. , select the corresponding inspection target, configure the corresponding inspection indicators, and configure the corresponding indicator weights to form a complete inspection indicator in the inspection template. Multiple inspection indicators are combined into a set of inspection templates, which are applied to exclusive in one of the systems to be checked. Finally, the inspection system is used to inspect the metadata to form a brief report of the evaluation results, including the sum of the evaluation results of the individual indicators and the product of their weights, and the general situation. evaluation result.

[0021] Finally, based on the results of the above three steps, a knowledge base of data quality issues is constructed, and historical records are managed to facilitate later reference.

[0022] It is worth noting that although the foregoing content has described the spirit and principle of the present invention with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that these Features in aspects cannot be combined, this division is for convenience of presentation only. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.