Method and device for generating data wide table in real time
A technology for producing data and data, which is applied in the computer field, can solve the problems of consuming more and more computing resources and time, and can not respond to data in real time, so as to achieve the effect of solving computing resources and computing time.
Active Publication Date: 2021-09-03
BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF9 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0006] The final GDM layer-wide table calculates the data of the previous day every day, and the data obtained by the user is T+1 data, which cannot reflect the data situation of the day in real time; with the sharp increase in the amount of data, the consumption is calculated once every morning in a fixed time period More and more computing resources and time
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0030] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.
[0031] figure 2 It is a schematic diagram of the main flow of the method for generating a data wide table in real time according to an embodiment of the present invention, such as figure 2 As shown, the method includes:
[0032] Step S201: Real-time storage of online production data;
[0033] Step S202: determine the field information contained in the data width table to ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a method and a device for generating wide data tables in real time, and relates to the technical field of computers. A specific implementation of the method includes: storing online production data in real time; determining the field information contained in the wide data table to be generated; determining the data to be calculated from the stored production data according to the field information; Using streaming computing, the data to be calculated is loaded into the memory database table for real-time calculation to generate a wide data table. This implementation can provide real-time wide table data, directly reduce the external service time of the wide table in the data warehouse from T+1 delay to minutes or even lower, and can quickly convert complex offline SQL calculations into real-time calculations, and solve the problem of The problem of computing resources and computing time increases due to data growth.
Description
technical field [0001] The invention relates to the field of computer technology, and in particular, to a method and device for generating a data wide table in real time. Background technique [0002] A data warehouse is a collection of subject-oriented, integrated, relatively stable, and time-varying data to support decision-making in business management. The data in the data warehouse is subject-oriented, corresponding to the application-oriented traditional database. [0003] For reasons such as making the data structure clear, facilitating data lineage tracking, and reducing repetitive development, the data warehouse adopts a hierarchical structure for storage, and corresponding hierarchical task processing logic: the first layer is the buffer data memory (BDM), The online business system data snapshot, which saves detailed data on a daily basis; the second layer is the foundation data memory (FDM), which organizes detailed data according to business concepts; the third...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/248G06F40/18
CPCG06F40/18
Inventor 袁建军刘业辉王彦明
Owner BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD