Data query method and device, electronic equipment and computer readable storage medium
By implementing data query methods on the database server, generating query keys and logical relational data, and supporting high concurrent queries in columnar storage, the problem of low data query efficiency in the existing technology is solved, and efficient and low-cost data query effect is achieved.
Patent Information
- Application Number
- CN202311605592.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-28
- Publication Date
- 2025-05-30
AI Technical Summary
In the prior art, data query efficiency is low, especially performance problems between analytical and business-based queries, resulting in increased system complexity, data inconsistency and high maintenance costs.
Implement a data query method on the database server, which generates query keys and logical relational data by receiving the client's query request, and returns it to the client. When there is a query key in the target cache area, the query result is determined based on the query request and logical relational data and sent to the client. This method is suitable for columnar storage, supports high concurrent data queries, and reduces costs.
It improves the efficiency of data query, solves the performance problems of analytical and business-based queries, and meets the needs of multiple query scenarios, making data storage based on columnar storage, while achieving high concurrent queries, reducing system complexity and maintenance costs.
Smart Images
Figure CN120067150A_ABST
Abstract
Description
Background Art
[0002] The query of data depends on the way of data storage. The row-based storage method is beneficial to the business query of wide tables, and the column-based storage method is beneficial to the query of single-column analytical data. In the prior art, the double databases of column-based and row-based storage methods are respectively used to process the query requests of analytical and business types. The data is synchronized through the network during this process. However, the network transmission and communication costs are relatively high, the implementation and maintenance costs required are relatively high, the possibility of data inconsistency is increased, the complexity of the system is increased, and the query service of data analysis cannot be efficiently completed.
[0003] It should be noted that the information disclosed in the above background art section is only used to strengthen the understanding of the background of the present disclosure. Therefore, it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention
[0004] The present disclosure provides a data query method, device, electronic device, and computer-readable storage medium, which at least overcome the problem of low data query efficiency in the related art to a certain extent.
[0005] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be partially learned through the practice of the present disclosure.
[0006] According to an aspect of the present disclosure, there is provided a data query method, which is applied to a database server and includes: receiving a query request sent by a client; when the query key corresponding to the query request does not exist in the target buffer, generating the corresponding query key and logical relationship data in the target buffer, and returning the logical relationship data to the client; wherein the logical relationship data is column data arranged in a row form; when the query key corresponding to the query request exists in the target buffer, determining a query result according to the query request and the logical relationship data corresponding to the query key; and sending the query result to the client so that the client can parse the logical relationship data and the query result to obtain an actual query result.
[0007] In an embodiment of the present disclosure, the step of when the query key corresponding to the query request does not exist in the target buffer, generating the corresponding query key and the logical relationship data in the target buffer, and returning the logical relationship data to the client includes: when the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data exists in the column buffer, storing the pre-query data in the target buffer; and processing the pre-query data in the target buffer to generate the corresponding query key and the logical relationship data.
[0008] In one embodiment of the present disclosure, when the query key corresponding to the query request does not exist in the target buffer, generating the corresponding query key and the logical relationship data in the target buffer and returning the logical relationship data to the client includes: when the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer and the pre-query data does not exist in the column buffer, loading the pre-query data from the storage system and storing it in the target buffer; processing the pre-query data in the target buffer to generate the corresponding query key and the logical relationship data.
[0009] In one embodiment of the present disclosure, the pre-query data in the target buffer is processed by at least one of the following methods:
[0010] Merging the pre-query data with the same association relationship;
[0011] Merging the pre-query data with similar association relationships;
[0012] When at least one column data in the pre-query data is not queried within the first time period, deleting the column data and reorganizing the pre-query data;
[0013] Splitting the pre-query data into an index column and a sub-list corresponding to the index column.
[0014] In one embodiment of the present disclosure, the query result includes at least one of: query parameters, the actual query result, and the logical relationship data.
[0015] In one embodiment of the present disclosure, the method further includes: when the query request is an analytical query request, obtaining the actual query result corresponding to the query request from the column buffer.
[0016] In one embodiment of the present disclosure, it further includes: deleting the data in the actual query result that has not been queried within the second time period.
[0017] According to another aspect of the present disclosure, there is also provided a data query device, including:
[0018] A request receiving module, which receives a query request sent by a client;
[0019] A first generation module, when the query key corresponding to the query request does not exist in the target buffer, generating the corresponding query key and logical relationship data in the target buffer and returning the logical relationship data to the client; wherein, the logical relationship data is column data arranged in a row format;
[0020] A second generation module, when there is a query key corresponding to the query request in the target buffer, determines a query result according to the query request and the logical relationship data corresponding to the query key; and sends the query result to the client, so that the client can parse the logical relationship data and the query result to obtain an actual query result.
[0021] According to another aspect of the present disclosure, there is also provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the data query method according to any one of the above by executing the executable instructions.
[0022] According to another aspect of the present disclosure, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the data query method according to any one of the above is implemented.
[0023] The data query method, device, electronic device and computer-readable storage medium provided by the embodiments of the present disclosure receive a query request sent by a client; when there is no query key corresponding to the query request in the target buffer, a corresponding query key and logical relationship data are generated in the target buffer, and the logical relationship data is returned to the client; when there is a query key corresponding to the query request in the target buffer, query results such as query parameters are determined according to the query request and the logical relationship data corresponding to the query key; the query result is sent to the client, so that the client can parse the logical relationship data and the query result to obtain an actual query result; if the query request is an analytical query request, the actual query result corresponding to the query request is obtained from the column buffer; it can solve the performance problems of current analytical and business queries, meet the requests of various query scenarios, enable data storage to be based on columnar storage, and can also complete high-concurrency data query operations, improve the efficiency of data query, and reduce costs.
[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.
[0026] Figure 1 Shows a flowchart of a data query method in an embodiment of the present disclosure;
[0027] Figure 2 Shows a flowchart of a method for generating query keys and logical relationship data in an embodiment of the present disclosure;
[0028] Figure 3 Shows a flowchart of an analytical query request processing method in an embodiment of the present disclosure;
[0029] Figure 4 Shows a flowchart of a pre-query logic mechanism in an embodiment of the present disclosure;
[0030] Figure 5 Shows a schematic diagram of data query in an embodiment of the present disclosure;
[0031] Figure 6 Shows a schematic diagram of a data query device in an embodiment of the present disclosure;
[0032] Figure 7 Shows a schematic diagram of an exemplary system architecture to which the data query method or data query device in the embodiments of the present disclosure can be applied; and
[0033] Figure 8 Shows a structural block diagram of an electronic device in an embodiment of the present disclosure. Detailed implementation manners
[0034] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and thorough, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.
[0035] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0036] The following will describe the present example embodiment in detail with reference to the accompanying drawings and embodiments.
[0037] First, an embodiment of the present disclosure provides a data query method, which can be executed by any electronic device with computing and processing capabilities.
[0038] Figure 1 Shows a flowchart of a data query method in an embodiment of the present disclosure, asFigure 1 As shown, the data query method provided in the embodiment of the present disclosure is applied to the database server, and includes the following steps:
[0039] S102: Receive a query request sent by a client.
[0040] In one embodiment, when a business system is started and / or changed, the business system initiates a query request to a database client, and the client forwards the query request to a database server.
[0041] In one embodiment, the query request includes but is not limited to: at least one of the query column data, query conditions, etc.; for example, the query condition may be the sales data of region A, and the query column data may be the first column data of table 1, etc.
[0042] In one embodiment, the types of query requests include, but are not limited to: analytical query requests, business query requests, etc.
[0043] Analytical query requests are aimed at finding the sum, maximum, or minimum of a column, and do not involve table associations; business query requests require more associations between tables.
[0044] In one embodiment, the database is stored in column format, with data stored column by column at the bottom layer. Frequently used column data is placed in a column cache. If the query request is an analytical query request, the actual query result corresponding to the query request is obtained from the column cache, and processing is performed directly on a certain column. Column-based storage is more conducive to the execution of the query request and improves query efficiency.
[0045] S104: When the query key corresponding to the query request does not exist in the target cache area, the corresponding query key and logical relationship data are generated in the target cache area, and the logical relationship data is returned to the client.
[0046] In one embodiment, the logical relationship data is column data arranged in rows, index data, etc.; that is, the target cache area stores organized column data, arranged in rows, and accompanied by index data; the index data is used to identify which section of the data is within the range.
[0047] For the client, the query key is a string generated by the database server to identify the current query logic, that is, the string of logical relationship data. Each query key corresponds to a logical relationship data. Different query keys mean different strings of logical relationship data, and query keys and logical relationship data are only generated for business-type query requests. For analytical query requests, the column buffer can be directly queried. The query value corresponding to the query key is the logical relationship data, which is the metadata information explaining the pre-query result and the corresponding data structure. For example, which columns there are, which data in the columns is one-to-one, and which data in the columns is the set attribute of one-to-many. When the client actually queries a business-type query request next time, the server can return query parameters, and the client can associate the logical relationship data and the query parameters according to the query key and perform parsing to generate the actual query result entity.
[0048] For the database server, the query value corresponding to the query key is the pre-query logic, that is, the logical relationship data. For example, which columns are needed, how to organize them, cache data, etc. For example, there are 5 tables to be joined, with 50 fields serving a query. Then a query key will be generated according to this query. Among these 5 tables, 3 tables are one-to-one, with a total of 30, and 2 tables are one-to-many, with 10 each, for a total of 20. Then the 30 one-to-one fields involved will be joined first and placed in the target buffer, then the latter 20 fields will be added, and at the end, 2 fields will be appended to store the memory start address of the corresponding rows of the one-to-many tables, with 1 column for each table. And an index column will be generated for the 50-field table (a total of 52 columns, but 2 of them are hidden columns). The index column can be implemented in various ways, which will be described below.
[0049] In one embodiment, when the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data exists in the column buffer, the pre-query data is stored in the target buffer. When the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data does not exist in the column buffer, the pre-query data is loaded from the storage system and stored in the target buffer. The pre-query data in the target buffer is processed to generate the corresponding query key and logical relationship data.
[0050] S106. When the query key corresponding to the query request exists in the target buffer, determine the query result according to the query request and the logical relationship data corresponding to the query key.
[0051] In one embodiment, a query key is associated with a database session. When under the same database session, it is possible to access the logical relationship data multiple times to determine the query result. For the query of a business-type query request, two aspects of information can be extracted: logical relationship data and the actual query result. Extracting the logical relationship data can notify the database server in advance to prepare the columns for the wide-table business query. The database server assigns a query key for the pre-query business-type query request, associates the query key with the database session at the same time, and returns the logical relationship data to the client. When under the same database session, the business-type query request can be actually queried multiple times, that is, the logical relationship data can be accessed multiple times to obtain the query result and send it to the client.
[0052] In one embodiment, a business-type query request requires more associations between tables. The association between tables is more suitable under the condition of row storage, but the database underlying layer uses columnar storage. The logical relationship data is column data arranged in row form. When pre-querying a business-type query request, a query key and the corresponding logical relationship data are generated. The logical relationship data includes the required column data, etc. When actually querying the business-type query request next time, based on the query key and the corresponding logical relationship data, the query result corresponding to the query condition of the business-type query request is found.
[0053] S108. Send the query result to the client so that the client can parse the logical relationship data and the query result to obtain the actual query result.
[0054] In one embodiment, the query result includes but is not limited to at least one of the query key, query parameters, actual query result, and logical relationship data. For example, for select A from table1,table2 where table1.t1 = table2.t1 and table1.k1 = 5, the previously pre-allocated query key is key001, and the query parameter is 5; the actual query result is the actual query result determined according to the query parameter and the logical relationship data.
[0055] In one embodiment, only the query parameters can be sent to the client so that the client can parse the query result and the previously received logical relationship data to obtain the actual query result; alternatively, the database server can parse the query result and the logical relationship data to obtain the actual query result and send it to the client; alternatively, the query parameters and the logical relationship data can be sent to the client, and the client can parse the query result and the logical relationship data to obtain the actual query result, etc. High-concurrency data query operations can be completed, the data query efficiency can be improved, and the cost can be reduced.
[0056] In the above embodiments, when the query key corresponding to the query request does not exist in the target buffer, a corresponding query key and logical relationship data are generated in the target buffer. The logical relationship data is column data, index data, etc. arranged in a row format. When the query key corresponding to the query request exists in the target buffer, the query result is determined according to the query request and the logical relationship data corresponding to the query key. If the query request is an analytical query request, the actual query result corresponding to the query request is obtained from the column buffer. It can solve the performance problems of current analytical and business queries, meet the requests of various query scenarios, enable data storage based on columnar storage, and can also complete high-concurrency data query operations, improve the efficiency of data query, and reduce costs.
[0057] Figure 2 The flowchart of a method for generating a query key and logical relationship data in an embodiment of the present disclosure is shown. As Figure 2 shown, the method for generating a query key and logical relationship data provided in the embodiment of the present disclosure is applied to a database server, and includes the following steps:
[0058] S202, when the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data exists in the column buffer, store the pre-query data in the target buffer.
[0059] The pre-query data is the required column data and corresponding index data, etc. The column buffer is a buffer for storing data such as column data and corresponding index data. When the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data exists in the column buffer, store the pre-query data in the target buffer, and generate a query key and logical relationship data. When actually querying a business query request, query parameters can be obtained through the query key and logical relationship data, and high-concurrency data query operations can be completed, improving the efficiency of data query.
[0060] S204, when the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data does not exist in the column buffer, load the pre-query data from the storage system and store it in the target buffer; store the pre-query data in the target buffer, and generate a query key and logical relationship data. When actually querying a business query request, query parameters can be obtained through the query key and logical relationship data, and high-concurrency data query operations can be completed, and the storage cost can be reduced.
[0061] In one embodiment, the storage system is a device for long-term data storage, including but not limited to: hard disk drives, solid state drives, network storage, etc.
[0062] S206, process the pre-query data in the target buffer to generate corresponding query keys and logical relationship data.
[0063] In one embodiment, by processing the pre-query data in the target buffer through at least one of the following methods, the storage cost can be reduced and the query efficiency can be improved:
[0064] 1. Merge the pre-query data with the same association relationship, so that multiple queries can share cached data such as index data and pre-query data;
[0065] 2. Merge the pre-query data with similar association relationships, so that multiple queries can share cached data such as index data and pre-query data;
[0066] 3. When at least one column data in the pre-query data is not queried within the first time period, delete the column data and reorganize the pre-query data;
[0067] In one embodiment, obtain the number of database sessions for querying column data. When the number of database sessions is zero within the first time period, delete the column data and reorganize the pre-query data to reduce the number of random memory accesses during each query.
[0068] In one embodiment, the specific value of the first time period can be set manually or automatically according to historical data, experience, etc.
[0069] 4. Split the pre-query data into an index column and a sub-list corresponding to the index column; in one embodiment, the sub-list can be divided into a one-to-one table association list, a one-to-many table association list, etc.; the index column is the index column of the one-to-many table association list, pointing to the memory start address of several corresponding rows of the one-to-many table; add a hash index, a btree index, etc. to the query key of the main table; taking the hash index as an example, when the query parameter arrives, first find the corresponding row address through the hash index, then copy the attributes of the one-to-one table of that row to the output target buffer, and then copy the multiple records pointed to by the index column to the output target buffer; return the logical relationship data to the database client.
[0070] In the above embodiments, obtain the required column data, etc. in the target buffer, column buffer, storage system, etc. to generate a query key and logical relationship data; the database server determines the query result through the query key and logical relationship data, sends the query result to the client, and the client parses the logical relationship data and the query result to obtain the actual query result, so that the data storage is based on columnar storage, and high-concurrency data query operations can be completed, improving the data query efficiency and reducing the cost.
[0071] Figure 3 Show a flowchart of an analytical query request processing method in an embodiment of the present disclosure, as Figure 3 As shown, the analytical query request processing method provided in the embodiment of the present disclosure is applied to the database server and includes the following steps:
[0072] S302, receive a query request sent by the client.
[0073] S304, when the query request is an analytical query request, obtain the actual query result corresponding to the query request from the column buffer; just process a certain column directly. Columnar storage is more conducive to the execution of this query request and improves query efficiency.
[0074] S306, delete the data in the actual query result that has not been queried within the second time period.
[0075] LRU (Least Recently Used replacement algorithm), when the memory occupied by data reaches a certain threshold, delete the data that has been least recently accessed based on LRU. For example, the data at the leftmost end of the hash linked list will be deleted, and then the new data will be inserted at the rightmost position, saving resources and reducing storage costs.
[0076] In one embodiment, based on the LRU algorithm, delete the column data in the actual query result that has not been queried within the second time period.
[0077] In the above embodiment, when the query request is an analytical query request, just process a certain column directly. Columnar storage is more conducive to the execution of this query request and improves query efficiency. Delete the data that has been least recently accessed based on LRU, saving resources and reducing storage costs.
[0078] Figure 4 Show a flowchart of a pre-query logic mechanism in an embodiment of the present disclosure, as Figure 4 shown, the pre-query logic mechanism provided in the embodiment of the present disclosure is applied to the database server side and includes the following steps:
[0079] S402, when the business system starts or changes, the business system sends a query request to the database client, and the client forwards it to the server.
[0080] S404, when the query request is a pre-query business type query request, the server prepares the pre-query data and allocates query keys.
[0081] S406, pre-load the pre-query data into the target buffer area, and organize the data structure of the data to be queried according to the query logic, that is, generate query keys and logical relationship data.
[0082] The optimization means of the pre-query mechanism are as follows:
[0083] In one embodiment, the pre-query data can be optimized based on the query key;
[0084] Merge similar queries and merge the same associations to allow multiple queries to share cached data such as index data. Index data is used to identify which part of the data is within the range. There are many ways to implement it, such as using hash or btree.
[0085] Columns used for queries are cached on demand. If a column is not referenced by any session, the column can be automatically removed from memory. At the same time, cached columns are reorganized by row to reduce the number of random memory accesses in each query.
[0086] Split the query into multiple SQL (Structured Query Language), one is a one-to-one table association, and the other is a one-to-many independent query; at the same time, add an index column for the one-to-many table, pointing to the memory start address of several (add several index pointers if there are several one-to-many tables) corresponding rows of the one-to-many table. Add a hash index to the query key of the main table. When the query parameter arrives, first find the corresponding row address through the hash index, then copy the one-to-one table attributes of the row to the output buffer, and then copy the multiple rows pointed to by the index column to the output buffer; return the logical relationship data to the database client; the logical relationship data includes but is not limited to: pre-query results, metadata of the pre-query results and other data contents (n one-to-one attributes, m collection sub-attributes, each containing p1..pk attributes), etc.
[0087] S408, when the business system performs an actual query business query request, the query key and query parameters are directly passed; for example, select A from table1,table2 where table1.t1=table2.t1 and table1.k1=5, the previously pre-assigned query key is key001, and the query parameter is 5; the same applies to multiple parameters.
[0088] In the above embodiment, the database server determines the query result through the query key and logical relationship data, and sends the query result to the client. The client parses the logical relationship data and the query result to obtain the actual query result, so that the data storage is based on column storage, and high-concurrency data query operations can be completed, thereby improving the efficiency of data query and reducing costs.
[0089] Figure 5 A schematic diagram of data query in an embodiment of the present disclosure is shown. Figure 5 As shown, when the business system is started or changed, the business system initiates a query request to the database client, and the database client forwards it to the database server.
[0090] When an analytical query request is executed, the column buffer is directly queried. The column buffer will be checked by the database system, load column data that has not been loaded into memory, and use the LRU (Least Recently Used replacement algorithm) method to eliminate unused columns.
[0091] For example, the column buffer includes the following column data: Column 1, Column 2, Column 3, Column 4, Column 5, Column 6, Column 7, Column 8, Column 9, Column 10; when executing an analytical query request, the data of Column 1, Column 2, Column 3, and Column 4 can be directly obtained from the column buffer.
[0092] When a pre-query business query request is executed, the business system sends query requests such as query columns and conditions to the database system. The database system will first check whether the columns to be queried exist in the target buffer. If not, it will look for them in the column buffer and copy them to the target buffer. If they are not in the column buffer either, it will load the columns to be queried from the storage system and generate query keys and logical relationship data.
[0093] When an actual query business query request is executed, the query keys, logical relationship data, etc. are directly obtained from the target buffer, and the query results are determined.
[0094] For example, the column buffer includes the following column data: Column 1, Column 2, Column 3, Column 4, Column 5, Column 6, Column 7, Column 8, Column 9, Column 10; when executing a pre-query business query request, the query keys and logical relationship data corresponding to Column 6, Column 7, and Column 8 are generated and stored in the target buffer; when an actual query business query request is executed based on the same database session, the query keys and logical relationship data corresponding to Column 6, Column 7, and Column 8 are in the target buffer, and query results such as query parameters are returned to the client.
[0095] In the above embodiments, when the query key corresponding to the query request does not exist in the target buffer, the corresponding query key and logical relationship data are generated in the target buffer. The logical relationship data is column data, index data, etc. arranged in row form. When the query key corresponding to the query request exists in the target buffer, the query results are determined according to the query request and the logical relationship data corresponding to the query key. If the query request is an analytical query request, the actual query results corresponding to the query request are obtained from the column buffer; it can solve the performance problems of current analytical and business queries, meet the requests of various query scenarios, enable data storage based on columnar storage, and can also complete high-concurrency data query operations, improve the efficiency of data query, and reduce costs.
[0096] Based on the same inventive concept, an embodiment of the present disclosure also provides a data query device as described in the following embodiments. Since the principle of solving problems in this device embodiment is similar to that of the above method embodiment, the implementation of this device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be elaborated.
[0097] Figure 6 A schematic diagram of a data query device in an embodiment of the present disclosure is shown. Figure 6 As shown, the data query device 6 includes: a request receiving module 601, a first generating module 602, a second generating module 603, a forwarding module 604, and a processing module 605;
[0098] The request receiving module 601 receives the query request sent by the client;
[0099] The first generation module 602 generates the corresponding query key and logical relationship data in the target cache area when the query key corresponding to the query request does not exist in the target cache area, and returns the logical relationship data to the client; wherein the logical relationship data is column data arranged in rows;
[0100] The first generation module 602 also includes a preprocessing module, which stores the pre-query data in the target cache area when the target cache area does not have the query key corresponding to the query request and the pre-query data in the query request, and the column cache area has the pre-query data; loads the pre-query data from the storage system and stores it in the target cache area when the target cache area does not have the query key corresponding to the query request and the pre-query data in the query request, and the column cache area does not have the pre-query data; processes the pre-query data in the target cache area to generate the corresponding query key and logical relationship data.
[0101] The second generation module 603, when the target cache area has a query key corresponding to the query request, determines the query result according to the query request and the logical relationship data corresponding to the query key; sends the query result to the client so that the client can parse the logical relationship data and the query result to obtain the actual query result.
[0102] The second generating module 603 also includes: a first sending module that sends only the query parameters to the client, so that the client can parse the query result and the previously received logical relationship data to obtain the actual query result.
[0103] The second generation module 603 also includes: a second sending module, which can parse the query result and the logical relationship data at the database server to obtain the actual query result and send it to the client.
[0104] The second generating module 603 also includes: a third sending module, which sends query parameters and logical relationship data to the client, and the client parses the query result and the logical relationship data to obtain the actual query result, etc.
[0105] The data query device 6 also includes a forwarding module 604. When the business system is started and / or changed, the business system initiates a query request to the database client, and the client forwards it to the database server.
[0106] The data query device 6 further includes a processing module 605. When the query request is an analytical query request, the actual query result corresponding to the query request is obtained from the column buffer area, and only the processing for a certain column needs to be directly performed. The columnar storage is more conducive to the execution of this query request, improving the query efficiency.
[0107] The processing module is further configured to delete the data that has not been queried within the second time in the actual query result.
[0108] In the above embodiments, when the query key corresponding to the query request does not exist in the target buffer area, the corresponding query key and logical relationship data are generated in the target buffer area. When the query key corresponding to the query request exists in the target buffer area, the query result is determined according to the query request and the logical relationship data corresponding to the query key. When the query request is an analytical query request, the actual query result corresponding to the query request is obtained from the column buffer area; it can solve the performance problems of current analytical and business-type queries, meet the requests of various query scenarios, enable the data storage to be based on columnar storage, and can also complete high-concurrency data query operations, improve the data query efficiency, and reduce the cost.
[0109] Figure 7 The figure shows a schematic diagram of an exemplary system architecture to which the data query method or data query device according to the embodiments of the present disclosure can be applied.
[0110] As Figure 7 shown, the system architecture 700 may include client devices 701, 702, 703, a network 704, and a database server 705.
[0111] The network 704 is a medium for providing a communication link between the client devices 701, 702, 703 and the database server 705, and can be a wired network or a wireless network.
[0112] Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network. In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can be used to encrypt all or some of the links. In other embodiments, customized and / or proprietary data communication technologies can also be used to replace or supplement the above data communication technologies.
[0113] The clients 701, 702, 703 can be various electronic devices, including but not limited to smartphones, tablets, laptop computers, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.
[0114] Optionally, the clients of the application programs installed in the different clients 701, 702, 703 are the same, or are clients of the same type of application program based on different operating systems. Depending on the different terminal platforms, the specific form of the client of the application program can also be different. For example, the client of the application program can be a mobile client, a PC client, etc.
[0115] The database server 705 can be a database server that provides various services. For example, it can be a background management database server that supports the devices operated by users using clients 701, 702, and 703. The background management database server 705 can analyze and process data such as requests received, and feedback the processing results to clients 701 / 702, 703. For example, the database server 705 receives query requests sent by clients 701 / 702, 703. When the query key corresponding to the query request does not exist in the target buffer, a corresponding query key and logical relationship data are generated in the target buffer, and the logical relationship data is returned to clients 701 / 702, 703. When the query key corresponding to the query request exists in the target buffer, query results such as query parameters are determined according to the query request and the logical relationship data corresponding to the query key. The query results are sent to clients 701 / 702, 703 so that clients 701 / 702, 703 can parse the logical relationship data and the query results to obtain the actual query results. If the query request is an analytical query request, the actual query results corresponding to the query request are obtained from the column buffer.
[0116] Optionally, the database server can be an independent physical database server, or a database server cluster or distributed system composed of multiple physical database servers. It can also be a cloud database server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the database server can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here.
[0117] Those skilled in the art can know that Figure 7 the numbers of clients, networks, and database servers in
[0118] are merely illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers. The embodiments of the present disclosure do not limit this.
[0119] Next, with reference to Figure 8Describe the electronic device 800 according to this embodiment of the present disclosure. Figure 8 The displayed electronic device 800 is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.
[0120] As Figure 8 shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810 described above, the at least one storage unit 820 described above, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).
[0121] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 810, so that the processing unit 810 executes the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of this specification above.
[0122] For example, the processing unit 810 may execute the following steps of the above method embodiment: receiving a query request sent by a client; when the query key corresponding to the query request does not exist in the target cache area, generating a corresponding query key and logical relationship data in the target cache area, and returning the logical relationship data to the client; when the query key corresponding to the query request exists in the target cache area, determining query results such as query parameters according to the query request and the logical relationship data corresponding to the query key; sending the query results to the client so that the client can parse the logical relationship data and the query results to obtain the actual query results; if the query request is an analytical query request, obtaining the actual query results corresponding to the query request from the column cache area.
[0123] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 8201 and / or a cache storage unit 8202, and may further include a read-only storage unit (ROM) 8203.
[0124] The storage unit 820 may further include a program / utilities 8204 having a set (at least one) of program modules 8205. Such program modules 8205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.
[0125] The bus 830 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any bus structure in a variety of bus structures.
[0126] The electronic device 800 can also communicate with one or more external devices 840 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 850. Moreover, the electronic device 800 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 860. As shown in the figure, the network adapter 860 communicates with other modules of the electronic device 800 through the bus 830. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0127] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
[0128] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium, which can be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementation manners, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.
[0129] For example, when the program product in the embodiments of the present disclosure is executed by a processor, a method with the following steps is implemented: receiving a query request sent by a client; when a query key corresponding to the query request does not exist in the target buffer, generating a corresponding query key and logical relationship data in the target buffer, and returning the logical relationship data to the client; when a query key corresponding to the query request exists in the target buffer, determining query results such as query parameters according to the query request and the logical relationship data corresponding to the query key; sending the query results to the client so that the client can parse the logical relationship data and the query results to obtain the actual query results; if the query request is an analytical query request, obtaining the actual query results corresponding to the query request from the column buffer.
[0130] More specific examples of the computer-readable storage medium in the present disclosure may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0131] In the present disclosure, the computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, and this readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.
[0132] Optionally, the program code contained on the computer-readable storage medium may be transmitted by any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.
[0133] In specific implementation, program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).
[0134] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0135] In addition, although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution, etc.
[0136] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of the present disclosure.
[0137] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.
Claims
1. A data query method, characterized in that, applied to a database server, including: receiving a query request sent by a client; when the query key corresponding to the query request does not exist in the target buffer, generating the corresponding query key and logical relationship data in the target buffer, and returning the logical relationship data to the client; wherein, the logical relationship data is column data arranged in a row format; when the query key corresponding to the query request exists in the target buffer, determining a query result according to the query request and the logical relationship data corresponding to the query key; sending the query result to the client, so that the client can parse the logical relationship data and the query result to obtain an actual query result.
2. The data query method according to claim 1, characterized in that, the step of when the query key corresponding to the query request does not exist in the target buffer, generating the corresponding query key and the logical relationship data in the target buffer, and returning the logical relationship data to the client includes: when the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data exists in the column buffer, storing the pre-query data in the target buffer; processing the pre-query data in the target buffer to generate the corresponding query key and the logical relationship data.
3. The data query method according to claim 1, characterized in that, the step of when the query key corresponding to the query request does not exist in the target buffer, generating the corresponding query key and the logical relationship data in the target buffer, and returning the logical relationship data to the client includes: when the query key corresponding to the query request and the pre-query data in the query request do not exist in the target buffer, and the pre-query data does not exist in the column buffer, loading the pre-query data from the storage system and storing it in the target buffer; processing the pre-query data in the target buffer to generate the corresponding query key and the logical relationship data.
4. The data query method according to claim 2 or 3, characterized in that, processing the pre-query data in the target buffer by at least one of the following methods: merging the pre-query data with the same association relationship; merging the pre-query data with similar association relationships; when at least one column data in the pre-query data is not queried within a first period of time, deleting the column data and reorganizing the pre-query data; splitting the pre-query data into an index column and a sub-list corresponding to the index column.
5. The data query method according to claim 1, characterized in that, the query result includes at least one of: query parameters, the actual query result, and the logical relationship data.
6. The data query method according to claim 1, characterized in that, the method further includes: if the query request is an analytical query request, obtaining the actual query result corresponding to the query request from the column buffer.
7. The data query method according to claim 6, wherein, it further comprises: deleting the data in the actual query result that has not been queried within the second time period.
8. A data query device, wherein, it comprises: a request receiving module, configured to receive a query request sent by a client; a first generating module, configured to generate the corresponding query key and logical relationship data in the target buffer when the query key corresponding to the query request does not exist in the target buffer, and return the logical relationship data to the client; wherein, the logical relationship data is column data arranged in a row format; a second generating module, configured to determine a query result according to the query request and the logical relationship data corresponding to the query key when the query key corresponding to the query request exists in the target buffer; and send the query result to the client, so that the client can parse the logical relationship data and the query result to obtain an actual query result.
9. An electronic device, wherein, it comprises: a processor; and a memory, configured to store executable instructions of the processor; wherein, the processor is configured to execute the data query method according to any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium, on which a computer program is stored, wherein, the computer program, when executed by a processor, implements the data query method according to any one of claims 1 to 7.