High-speed data query platform based on shared memory

Abstract

The invention discloses a high-speed data query platform based on a shared memory, and particularly relates to the field of computer data storage and retrieval. The high-speed data query platform comprises a console tool, the connecting end of the console tool is provided with the shared memory, the connecting end of the shared memory is provided with a data channel, and the input end of the shared memory is connected with a client through an API. The shared memory is used as a storage carrier; after data loading completion, the API provided by the platform is directly used for accessing the data on the shared memory according to a set algorithm; for an occasion requiring frequent data query in a project, the data query efficiency of the application program is greatly improved, the pressure problem of database data access does not need to be considered during application program programming, the data is cached to a local memory and then managed, and a real high-speed data query serviceis provided for an application client.

Description

technical field [0001] The invention relates to the field of computer data storage and retrieval, more specifically, the invention relates to a high-speed data query platform based on shared memory. Background technique [0002] With the continuous development of informatization, the speed of data generation is increasing, and the amount of data that needs to be processed is rapidly expanding. The era of big data is coming. The so-called big data refers to the huge amount of data involved, so that it cannot be processed in a reasonable way by mainstream software. In the face of massive data, although traditional relational databases have the advantages of supporting integrity constraints and supporting transactions, they are powerless in the face of large-scale massive data. Traditional relational databases mainly have the following problems. Data format conversion and storage cannot meet the performance requirements of massive data processing; second, it cannot meet the req...

AI Technical Summary

Problems solved by technology

[0002] With the continuous development of informatization, the speed of data generation is increasing, and the amount of data that needs to be processed is rapidly expanding. The era of big data is coming. The so-called big data refers to the huge amount of data involved, so that it cannot be processed in a reasonable way by mainstream software. In the face of massive data, although traditional relational databases have the advantages of supporting integrity constraints and supporting transactions, they are powerless in the face of large-scale massive data. Traditional relational databases mainly have the following problems. Data format conversion and storage cannot meet the performance requirements of massive data processing; second, it cannot meet the requirements of dynamic expansion and high availability; third, traditional large-scale relational databases usually run on large equipment, which is expensive, and the storage of data by big data How to effectively store, query and analyze these large-scale data has become an urgent problem to be solved

[0003] In terms of high-speed data storage and retrieval, there are currently many open source products on the market, which can provide relatively high-speed data storage and retrieval efficiency and relatively high concurrent visits, but all products currently on the market are basically based on the network. In the service mode, the query efficiency of data will inevitably be affected by protocol codec conversion and the delay of network communication itself. Even the in-memory database service known for its performance, the number of interactions per second for a single client frame is only one at most. At the same time, for requests from different clients, the server needs to queue up for processing, which cannot achieve concurrent access in the actual sense. In some occasions that require very high-speed access to data, for example, the application client needs frequent When requesting data and the client has strict requirements on the delay of calling and accessing the data interface, these products appear to be powerless

Images