Data import verification method, system and device and computer readable medium

A data verification and data import technology, applied in the field of data warehousing, can solve problems such as affecting database performance, less application, and inconvenient processing of database constraints, achieving high reliability and high availability, improving response speed, and realizing data visualization processing effect

Pending Publication Date: 2022-03-01
INSPUR SOFTWARE CO LTD
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Setting more constraints in the database will affect the performance of the database to a certain extent. It is rarely used in the real environment, and it is more processed in the program logic. It is possible that in the face of business changes or system expansion, the database Constraints can make processing inconvenient

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data import verification method, system and device and computer readable medium
  • Data import verification method, system and device and computer readable medium
  • Data import verification method, system and device and computer readable medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0056] The data import verification method of the present invention comprises the following steps:

[0057] S100. Configure basic constraints and association constraints. The basic constraints are used to constrain the fields in a single table, and the association constraints cooperate with the Es search engine to determine the relationship between the associated tables in the Es search engine based on the table structure relationship between the associated tables. The index relationship, and import the table data into the Es search engine in sequence according to the index relationship;

[0058] S200. Construct a data verification model based on basic constraints and management constraints, establish a physical index corresponding to the data verification model in the Es search engine, and simultaneously generate an excel template file corresponding to the data verification model, and the data verification model can be exported Excel files can be saved and imported into excel...

Embodiment 2

[0091] The data import verification system of the present invention includes a constraint configuration module, a model building module, a data verification module, and a data storage module. The constraint configuration module is used to configure basic constraints and associated constraints, and the basic constraints are used to check fields in a single table Carry out constraint, described association constraint cooperates with Es search engine, is used for determining the index relationship of described association table in Es search engine based on the table structure relationship between association tables, and imports table data into Es search sequentially according to index relationship In the engine; the model building module is used to build a data verification model based on basic constraints and management constraints, establishes a physical index corresponding to the data verification model in the Es search engine, and generates synchronously corresponding to the da...

Embodiment 3

[0097] The device of the present invention includes: at least one memory and at least one processor; at least one memory for storing a machine-readable program; at least one processor for calling the machine-readable program to execute the method disclosed in Embodiment 1 .

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data import verification method, a data import verification system, a data import verification device and a computer readable medium, belongs to the technical field of data storage, and aims to solve the technical problem of how to realize integrity verification of data import and improve data reliability and system availability. S100, configuring a basic constraint and an association constraint; s200, constructing a data verification model based on the basic constraint and the management constraint, establishing a physical index corresponding to the data verification model in an Es search engine, and synchronously generating a corresponding excel template file; s300, storing excel data conforming to the constraint into an Es search engine, storing excel data which does not conform to the constraint as failure data, and prompting a detection failure reason; and S400, inserting the excle data stored in the Es search engine into the database, performing excel export and correction on the failure data, and executing the steps S300-S400 on the corrected failure data.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a data import verification method, system, device and computer readable medium. Background technique [0002] In the field of government service products, there are many scenarios where it is necessary to import historical data or system data that cannot be connected, but each business system has its own business rules, which leads to certain constraints on the imported data. There is also a case of unqualified data in the database. [0003] In view of the above situation, and considering the requirements of strong data correlation, strict format, and huge data volume in business scenarios, it is inevitable to use a suitable model for data inspection when importing data, and for the case of large volume of imported data Adaptation is required. [0004] The integrity of the database refers to the correctness and compatibility of the data, and the integrity control of the da...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215G06F16/22G06F16/25
CPCG06F16/2228G06F16/215G06F16/258
Inventor 倪祥路
Owner INSPUR SOFTWARE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap