Public layer wide table construction method and device and server

A public-layer, wide-table technology, applied in the field of data processing, can solve the problems of timeliness and business statistics complexity, unfavorable data structure, expansion of international business overall data insights, etc., to ensure correct output and improve data quality Effect

Active Publication Date: 2019-12-06
ADVANCED NEW TECH CO LTD
View PDF9 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Half of the offline data warehouse construction time is T+1, that is, if you want to see the detailed data and summary data of yesterday's whole day before 8:00 this morning, you can start ETL on yesterday's business data at 0:00 this morning processing, but for business data in other time zones, ETL processing will start later than 0:00 a.m.
In a multi-time zone data scenario, if the scheduling time and business statistics time matching each time zone are established, it will be difficult to meet the current business demands in terms of timeliness and business statistics complexity, and it is also not conducive to the expansion of data architecture and Holistic data insights for international business

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Public layer wide table construction method and device and server
  • Public layer wide table construction method and device and server
  • Public layer wide table construction method and device and server

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach 1

[0044] Embodiment 1 specifically includes the following steps:

[0045] Step 1. Before the sub-incremental data is entered into the target database, the sub-incremental data generated by each sub-business table is merged and / or the rows of the data records are screened to obtain the target warehousing data, and the target warehousing data The data records of are in one-to-one correspondence with the data records in the main incremental data.

[0046] Specifically, before the sub-incremental data of each sub-business table is entered into the target database, according to the business logic in the business system, the sub-incremental data is merged and / or data-recorded. row filter.

[0047] Among them, the row merging of data records is: multiple rows of data records in the sub-incremental data generated by each sub-business table are merged into one row of data records, wherein, the multiple rows of data in the sub-incremental data need to be merged into one row of data recor...

Embodiment approach 2

[0057] Embodiment 2: The primary key value of each row of data records in the primary incremental data is spliced ​​with the primary key value of the corresponding data record in the secondary incremental data to form the associated primary key value of each row of data records in the secondary incremental data. Specifically, you can use commas or other symbols to concatenate primary key values. Through the second embodiment, the process of data record merging and screening can be avoided, and the timeliness is higher.

[0058] If the second embodiment is used to design the associated primary key value, then S202 specifically includes:

[0059]Step 2021, according to the primary key value of each row of data records in the primary incremental data and its corresponding associated primary key value, determine in the hot data table related to the main business table that it matches the data record in the primary incremental data The full amount of hotspot data records.

[0060...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a public layer wide table construction method and device and a server. The public layer wide table construction method includes the steps: obtaining main incremental data in the main service table according to a preset delay duration in a delay manner, and waiting for arrival of auxiliary incremental data in the auxiliary business table by utilizing the preset delay duration, thus guaranteeing that all data needing to be associated in the auxiliary business table reach, and each row of data records of the main business table can be successfully matchedwith hotspot data in the auxiliary business table, guaranteeing that complete data can be associated successfully so as to avoid drifting of the business data, and guaranteeing correct output of the hour-level public layer wide table.

Description

technical field [0001] The embodiments of this specification relate to the field of data processing, and in particular to a method, device, and server for constructing a public layer wide table. Background technique [0002] At present, international trade is becoming more and more popular, and the development of international business will cover many countries, and each country has its own time zone. Half of the offline data warehouse construction time is T+1, that is, if you want to see the detailed data and summary data of yesterday's whole day before 8:00 this morning, you can start ETL for yesterday's business data at 0:00 this morning processing, but for business data in other time zones, ETL processing will start later than 0:00 AM. In a multi-time zone data scenario, if the scheduling time and business statistics time matching each time zone are established, it will be difficult to meet the current business demands in terms of timeliness and business statistics comp...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/25
CPCG06F16/2282G06F16/254
Inventor 李文启丁和生
Owner ADVANCED NEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap