System and method for achieving metadata analysis based on distributed cache

Abstract

The invention relates to a system and method for achieving metadata analysis based on distributed cache. The system comprises a metadata analysis client-side, a metadata analysis server-side, a distributed cache framework server and a distributed cache server. The method comprises the steps that the distributed cache framework server loads data in a metadatabase into the distributed cache server and creates a connection pool; the metadata analysis client-side sends a corresponding metadata analysis message to the metadata analysis server-side; the metadata analysis server-side inquires the distributed cache server according to a metadata analysis task which is analyzed and sends a corresponding analysis result message to the metadata analysis client-side; the metadata analysis client-side analyzes the analysis result message and displays a metadata analysis result. By means of the system and method for achieving metadata analysis based on distributed cache, data access performance is improved, the response speed of metadata analysis is increased, user experience is improved, and the system and method have a wider application range.

Description

technical field [0001] The present invention relates to the field of metadata analysis, in particular to the field of distributed cache and analysis of metadata, and specifically refers to a system and method for realizing metadata analysis based on distributed cache. Background technique [0002] With the development of information technology, massive amounts of data are generated, how to effectively manage and organize these massive amounts of data has become a prominent problem. Metadata is an effective means to solve this problem and can support the management and maintenance of data by the system. Specifically, in the data warehouse system, the metadata mechanism mainly supports the following five types of system management functions: [0003] (1) Describe what data is in the data warehouse; [0004] (2) Define the data to be entered into the data warehouse and the data generated from the data warehouse; [0005] (3) Record the working schedule of data extraction acc...

AI Technical Summary

Problems solved by technology

In this process, multiple SQL (Structured Query Language, Structured Query Language) statements need to be executed, and often the query condition of the latter SQL statement is the result obtained from the previous SQL statement, so these SQL statements can only be executed sequentially, not concurrently. , and the query operation of the database requires a large number of disk I / O (input / output) operations, the disk operation is very time-consuming, and the generation of metadata association graphs usually requires a long response time

Moreover, the analysis of metadata often involves the connection operation of multiple tables, and the connection operation of multiple tables consumes computer resources such as CPU and memory. If a large amount of data is stored in the table, the response time of data query will be slow very slow

[0010] The usual database optimization technology is to put the query results in the local cache to improve performance, but the local cache capacity is limited and cannot store a large amount of data. In addition, only when the query conditions are exactly the same as those stored in the local cache. Use cached data, otherwise you must query the database

Images