Data consistency checking method and device, electronic equipment and storage medium

An inspection method and consistency technology, applied in the field of financial technology, can solve problems such as data inconsistency and inability to check missing data, and achieve the effect of improving inspection quality

Pending Publication Date: 2022-05-10
INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Embodiments of the present invention provide a method for checking data consistency and its device, electronic equipment, and storage media, so as to at least solve the problem in the related art that it is impossible to check the lack of data in data collection and data transmission, which may easily cause data inconsistency question

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data consistency checking method and device, electronic equipment and storage medium
  • Data consistency checking method and device, electronic equipment and storage medium
  • Data consistency checking method and device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] According to an embodiment of the present invention, an embodiment of a data consistency checking method is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and , although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0049] figure 1 is a flow chart of an optional data consistency checking method according to an embodiment of the present invention, such as figure 1 As shown, the method includes the following steps:

[0050] Step S101, obtain the lake data recorded in the target data lake and the source data transmitted to the target data lake, wherein the source data includes at least: the source system table, the number of source table records collected by each data source application is pre-recorded in the source sys...

Embodiment 2

[0124] The present application will be described below in conjunction with a preferred data consistency check method.

[0125] Figure 7 is a schematic diagram of an optional data lake and data source consistency check quality inspection device according to an embodiment of the present invention, as shown in Figure 7 As shown, it includes: data source application, data lake application, metadata management platform, and inspection device. By inputting the detected information in the data source application and data lake application to the inspection device, the inspection device uses the reference input from the metadata management platform Based on the information, check the consistency between the data lake and the data source, and feed back the detection information to the data source application and the data lake application. The specific check of the consistency between the data lake and the data source mainly includes data file format check, each 5 checks including dai...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data consistency checking method and device, electronic equipment and a storage medium, and relates to the field of financial science and technology, and the checking method comprises the steps: obtaining lake data recorded in a target data lake and source data transmitted to the target data lake, and transmitting source data to the target data lake when a source file corresponding to the source data is transmitted; counting the total file record number of the source file, checking whether the total file record number is consistent with the loading record number or not to obtain a first checking result, checking whether the source table record number is consistent with the loading record number or not to obtain a second checking result, and determining that the lake data is consistent with the source data under the condition that the first checking result and the second checking result indicate that checking is consistent. According to the method and the device, the technical problem of data inconsistency easily caused by incapability of checking data missing in data acquisition and data transmission in related technologies is solved.

Description

technical field [0001] The present invention relates to the field of financial science and technology, in particular, to a method for checking data consistency, a device thereof, electronic equipment, and a storage medium. Background technique [0002] A data lake is a large-scale warehouse that stores a variety of raw data. It can obtain raw data from multiple business systems of an organization and store the data in a centralized manner in the form of copies. The data lake is the basic data support of the entire data center, and efficient quality assurance is imperative. Therefore, whether the data quality inspection mechanism of the data lake is perfect will have a crucial impact on the value of data use. [0003] In related technologies, data lake data quality assurance has the following two defects: [0004] (1) Guarantee effect: There are many technical links such as data collection, transmission and loading between the data lake and the business system, and it is nec...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/22G06F16/215
CPCG06F16/2365G06F16/2282G06F16/215
Inventor 陈雪姗
Owner INDUSTRIAL AND COMMERCIAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap