A data acquisition method, device and system for a data warehouse
A data warehouse and data acquisition technology, applied in the field of data processing, can solve the problems of server performance deterioration and time lag in the data warehouse, and achieve the effect of improving efficiency and avoiding server performance deterioration
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
no. 1 example
[0024] figure 1 It is an implementation flowchart of a data collection method for a data warehouse provided in the first embodiment of the present invention. The method can be executed by a data collection device for a data warehouse, where the device can be implemented by software and / or hardware, and can be used as a data Part of the warehouse is built inside the data warehouse. Such as figure 1 As shown, the implementation process includes:
[0025] Step 11, periodically detecting whether a preset data extraction event in at least one database server connected to the data warehouse is triggered.
[0026] Among them, the data warehouse, as a factory for data collection, data processing, and data output, supports various data requirements such as data analysis, reporting, and mining, and plays the role of data-driven value. The warehouses used to store commodities are distributed all over the country, and each warehouse independently deploys the same WMS (Warehouse Managem...
no. 2 example
[0040] On the basis of the foregoing embodiments, this embodiment provides a data collection method for a data warehouse.
[0041] figure 2 and image 3 All are flowcharts of realizing a data collection method of a data warehouse provided in the second embodiment of the present invention. combine figure 2 and image 3 , the data acquisition method of the data warehouse includes:
[0042] Step 21, the task scheduling unit in the data warehouse creates a data extraction task.
[0043] Still taking the connection between the data warehouse and the database servers corresponding to M warehouses as an example, the task scheduling unit creates M tasks, and each task is used to extract data from N tables in a database server.
[0044] Step 22, setting a time threshold.
[0045] For the task created by the task scheduling unit, set the time threshold (ie the latest extraction time), and the database server that has not been extracted by the time threshold will start to execute...
no. 3 example
[0057] Figure 4 It is a schematic structural diagram of a data collection device for a data warehouse provided in the third embodiment of the present invention, and the device can be built in the data warehouse. Such as Figure 4 As shown, the data acquisition device of the data warehouse includes a data extraction detection unit 31 and a data extraction storage unit 32 .
[0058] Wherein, the data extraction detection unit 31 is used to regularly detect whether the preset data extraction event in at least one database server connected to the data warehouse is triggered;
[0059] The data extraction storage unit 32 is configured to extract and store data in any one of the database servers when it is detected that a data extraction event in the database server is triggered.
[0060] Optionally, the data extraction detection unit 31 includes:
[0061] The check-in detection subunit is used to regularly detect whether the check-in event in each of the database servers is trig...
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More