Method, device and system for data extraction based on remainder

A data extraction and data volume technology, applied in database management systems, structured data retrieval, special data processing applications, etc., can solve the problems of low extraction efficiency, short board effect, waste of human resources, etc., to save extraction time and prevent Short board effect, the effect of improving the timeliness of extraction

Active Publication Date: 2020-06-30
BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] 1) The extraction efficiency is low, and the data acquisition cycle is too long, which cannot meet the timeliness requirements of the data warehouse, affects the company's corresponding decision-making, and causes unnecessary losses to the company;
[0007] 2) It is necessary to manually adjust the extraction conditions from time to time, which wastes a lot of human resources and is prone to errors;
[0008] 3) Using segment extraction by date, the amount of data generated every day is unevenly distributed, resulting in inconsistencies in the total amount of data extracted by each segment, which is prone to short board effects

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device and system for data extraction based on remainder
  • Method, device and system for data extraction based on remainder
  • Method, device and system for data extraction based on remainder

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0035] figure 1 is a schematic diagram of main steps of a method for data extraction based on remainder according to an embodiment of the present invention.

[0036] Such as figure 1 As shown, the method for data extraction based on remainder in the embodiment of the present invention mainly includes the following steps:

[0037] Step S11: create a target table in the d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data extraction method, device and system based on modulo operation. Therefore, integrity of extracted data is ensured while extraction efficiency is increased. Extraction duration is saved and the requirement for timeliness of the data warehouse level is met. The method comprises the following steps: creating a target list in a data warehouse; determining the number of sections according to an extraction demand, setting number extraction tasks according to the number of sections and configuring number extraction templates for the number extraction tasks by modulo operation; parallely executing the number extraction tasks according to the number extraction templates to extract sectional data; and combining sectional data to the target list.

Description

technical field [0001] The present invention relates to the field of computer technology and software, in particular to a method, device and system for data extraction based on remainder. Background technique [0002] ETL is an important part of building a data warehouse. Users extract data from different data sources through the ETL stage. After conversion, cleaning, loading and other processing processes, the processed data is finally processed according to the pre-defined data warehouse model. loaded into the data warehouse. As the first link in the data warehouse ETL process, the efficiency of data extraction directly affects the timeliness of the entire data warehouse. [0003] The extraction strategy of the offline data warehouse is generally to correspond to a data source table according to an extraction task, and set corresponding rules through the scheduling platform to execute data extraction operations on time every day to meet the processing needs of the data wa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/25
CPCG06F16/254
Inventor 张宁刘业辉王彦明杨钦龙
Owner BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap