Method, device and electronic equipment for data table derivation processing

Abstract

The application relates to a data pushing method and device for datasheets and electronic equipment; the method comprises: acquiring scheduling task dependence relation data from a datasheet-receiving target system; ranking datasheets to be pushed according to the scheduling task dependence relation data; pushing the datasheets to the target system according to the ranking results. The data pushing method and device for datasheets and electronic equipment according to the technical scheme can provide improved data pushing efficiency and enable the imbalance problem in the use of datasheets to be solved.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular, to a method, device and electronic equipment for processing data table inferences. Background technique [0002] At present, after establishing a unified data warehouse within an enterprise, due to different business application requirements, multiple data marts with different business lines are often established. Data marts are a subset of enterprise-level data warehouses, which are mainly oriented to department-level businesses. , and only for a specific topic. In order to solve the contradiction between flexibility and performance, the data mart is a small departmental or workgroup-level data warehouse added to the data warehouse architecture. The data mart stores pre-calculated data for a specific user to meet the performance requirements of the user. Data marts can alleviate the bottleneck of accessing data warehouses to a certain extent. In addition, the ...

AI Technical Summary

Problems solved by technology

[0011] However, the above-mentioned technical solution has the following problems: firstly, because there is no push priority control method, a large number of data with low frequency of use or even useless data may be pushed with high priority, resulting in low system usage efficiency

For example: the "table queue" 202 in the data mart 201 has 100 tables, and the 20th to 50th tables are rarely or basically not used in the data sub-mart, so these tables in the past are counted first because the usage rate is low Or not used by subsequent tasks, resulting in inefficient use of the data submart system

In addition, since there is no data table push data processing device, the efficiency of the data mart subsystem using the data table may be unbalanced

For example: push a table A to all data sub-marts in the past, but only table A is used in the data sub-mart, and table A is not used in other data sub-marts. In this case, data sub-mart 1 can perform follow-up Relying on the task of table A, while other data sub-marts are still in a state of continuing to wait, for all data sub-marts, the overall efficiency of use is declining

[0012] All in all, the above technology does not consider the application value of tables, including the importance and popularity of tables, when transferring data between data warehouses to data marts or data marts to data sub-mart systems, and only relies on manual experience. There is no reasonable method to control the priority of pushing numbers, so when pushing numbers, it will cause the target system to wait for subsequent tasks and reduce the overall system efficiency.

Therefore, the above-mentioned related technical solutions cannot form effective methods and strategies when pushing data from the data warehouse to the data mart or from the data mart to the data sub-mart, so it will have an impact on the system efficiency, and it is impossible to achieve multiple Effective management of pushing numbers from data submarts

Images