Data quality detection method, device and equipment and computer readable storage medium

Abstract

The invention relates to the field of financial science and technology, and discloses a data quality detection method and device based on machine learning, equipment and a computer readable storage medium. The method comprises the following steps: acquiring historical business data, and constructing an early warning model according to the historical business data; when a data quality detection request is received, determining a service scene identifier corresponding to the data quality detection request, and obtaining service data corresponding to the service scene identifier; inputting the business scene identifier into the early warning model to obtain an early warning threshold corresponding to the business scene identifier, and comparing the business data with the early warning threshold; and if the service data is greater than the early warning threshold, outputting prompt information that the service data is abnormal. According to the invention, the data quality detection efficiency is improved, and the waste of human resources in the data quality detection process is reduced.

Description

technical field [0001] The present invention relates to the technical field of financial technology (Fintech), in particular to a data quality detection method, device, equipment and computer-readable storage medium. Background technique [0002] In recent years, with the rapid development of Internet financial technology (Fintech), more and more technologies (big data, distributed, blockchain, artificial intelligence, etc.) are applied in the financial field. [0003] The data volume and business volume of financial business are increasing exponentially. In order to ensure data quality, it is usually necessary to perform data quality detection and early warning on financial business data. Currently, when testing and early warning of financial business data quality, data testers first need to configure specific data information, and then According to the experience of the data testers, configure the early warning rules; if the rules configured by the data testers are invalid...

AI Technical Summary

Problems solved by technology

[0003] The data volume and business volume of financial business are increasing exponentially. In order to ensure data quality, it is usually necessary to perform data quality detection and early warning on financial business data. Currently, when testing and early warning of financial business data quality, data testers first need to configure specific data information, and then According to the experience of the data testers, configure the early warning rules; if the rules configured by the data testers are invalid, the data testers need to manually modify the configuration until the data quality and data warnings are finally obtained; the current method of data quality detection requires Manual configuration, big data scenarios often involve a large number of data fields, the configuration work is cumbersome and the workload is heavy, and the configuration of early warning rules in data quality inspection depends on business experience and data understanding, and a large amount of data analysis on historical data is required, which requires data testing The personnel have a high professional level, which leads to a serious waste of human resources in the data quality inspection process. In addition, if there are a large number of invalid alarms, a large number of alarm configurations need to be modified to converge the alarms, resulting in a long data quality inspection cycle.

Images