Multi-dimension data distribution method based on data distribution base

Abstract

The invention discloses a multi-dimension data distribution method based on a data distribution base and belongs to the technical field of power grid data processing. The method includes the steps that firstly, the service logic is analyzed, the logic relation between the service function and tables in a database is analyzed, horizontal or vertical distribution is determined, the overall data partition rule is determined according to the service analysis result, the vertical distribution strategy is suitable for being adopted for a simple logic system, the horizontal distribution strategy is suitable for being adopted for a complex logic system, then multi-dimension mixed distribution is determined, and the distribution strategy in the data distribution base is combined to be the best mixed distribution by combination with the service logic; the finest granularity of data blocks distributed in a multi-dimension mode is determined through the highest common divisor rule by analyzing the storage space of cluster nodes; data distribution is achieved finally. The multi-dimension data distribution method based on the data distribution base can achieve high-performance data query and analysis, and meets the requirement of users for the real-time performance of big data processing.

Description

technical field [0001] The invention belongs to the technical field of grid data processing, and more precisely, the invention relates to a multi-dimensional data distribution method based on a data distribution database. Background technique [0002] With the gradual deepening of the application of power enterprise business information systems, the amount of business data has increased sharply. At present, the data of multiple systems has reached hundreds of millions of records, and the performance and operating efficiency of information systems have gradually decreased. Memory, with the help of memory computing, can effectively improve the operating performance of the power information system, but the data distribution strategy adopts horizontal slicing, which is relatively simple, resulting in uneven data blocks after splitting, and load balancing cannot be achieved. The optimal distribution strategy seriously affects the operating efficiency of the system. [0003] Data...

AI Technical Summary

Problems solved by technology

[0012] For big data in the power industry, the data distribution strategy adopts a single storage strategy, which cannot achieve load balancing and affects system operating efficiency. The above distributed file system, column-oriented distributed database and other technologies have limitations and can only be used to a certain extent. alleviate the problem

The above method does not consider business logic. Although it can achieve data storage load balance, it is not suitable for object-oriented parallel computing systems, and its performance needs to be further improved.

Images