Data query method, device, equipment and storage medium

Abstract

The present application provides a data query method, device, equipment and storage medium. The method includes: generating a first query statement corresponding to a first query task from a client; if it is determined that there is no first query result of the first query task in the cache according to the first query statement, determining whether there is the first query task in the pre-processing task information configuration table in the first database according to the first query statement, the pre-processing task information configuration table records at least one pre-processed query task; if the first query task exists in the pre-processing task information configuration table, obtaining the first query result from the pre-processing result table corresponding to the first query task using the first query statement; and sending the first query result to the client. The method of the present application can improve query efficiency.

Description

Technical Field

[0001] The present application relates to the field of data query technology, and in particular to a data query method, device, equipment and storage medium. Background Art

[0002] Data query refers to a query technology that queries specified information from a database and feeds it back to the user based on user needs and certain algorithms. The data query system is functionally divided into a user layer, a service layer, and a data storage layer. The user layer includes a client on the user side, which is used to receive user query requests and provide the queried data to the user; the service layer is used to perform data query operations; and the data storage layer is used to store data.

[0003] In the prior art, after the user layer of the data query system receives the query request from the client, the service layer converts the request into a corresponding SQL statement so as to find the query result required by the request in the database of the data storage layer. The data storage layer is provided with a MySQL database and a hive database, and the data occupying a small storage space is stored in the MySQL database, and the data occupying a large storage space is stored in the hive database, so as to facilitate the query according to the size of the storage space occupied by the data.

[0004] With the growth and accumulation of massive data, there are more and more time-consuming data query tasks, which causes the data query system to spend more and more time to execute data query tasks, and even query timeouts occur, which in turn leads to lower and lower query efficiency of the data query system. Summary of the invention

[0005] The present application provides a data query method, apparatus, device and storage medium to solve the problem of increasingly low query efficiency of a data query system.

[0006] In a first aspect, the present application provides a data query method, the method comprising:

[0007] According to a first query task from a client, generating a first query statement corresponding to the first query task;

[0008] If it is determined according to the first query statement that there is no first query result of the first query task in the cache, then determining according to the first query statement whether the first query task exists in a preprocessing task information configuration table in the first database, wherein the preprocessing task information configuration table records at least one preprocessing query task;

[0009] If the first query task exists in the pre-processing task information configuration table, using the first query statement to obtain the first query result from the pre-processing result table corresponding to the first query task;

[0010] The first query result is sent to the client.

[0011] Optionally, after determining whether the first query task exists in the pre-processing task information configuration table in the first database, the method further includes:

[0012] If the first query task does not exist in the pre-processing task information configuration table, the first query statement is used to obtain the first query result from the source data table.

[0013] Optionally, the source data table is located in a second database.

[0014] Optionally, after generating the first query statement corresponding to the first query task, the method further includes:

[0015] If it is determined according to the first query statement that the first query result exists in the cache, the first query statement is used to obtain the first query result from the cache.

[0016] Optionally, generating a first query statement corresponding to the first query task according to the first query task from the client includes:

[0017] The multiple query conditions included in the first query task are concatenated and converted to obtain the first query statement.

[0018] Optionally, the method further comprises:

[0019] According to the first query statement, the first query result is stored in the cache.

[0020] Optionally, the method further comprises:

[0021] The first query information corresponding to the first query task is stored in the historical query information record table of the first database, where the first query information includes: the first query statement and query time sub-information of the first query task.

[0022] Optionally, before determining according to the first query statement that there is no first query result of the first query task in the cache, the method further includes:

[0023] Determine whether the historical query information record table has the first query task;

[0024] If it is determined that the historical query information record table has the first query task, determining whether the cache has the first query result of the first query task according to the first query statement;

[0025] or,

[0026] If it is determined that the historical query information record table does not have the first query task, using the first query statement to obtain the first query result from the source data table;

[0027] The first query result is sent to the client.

[0028] Optionally, the method further comprises:

[0029] According to a preset first screening condition, at least one second query task to be preprocessed is screened from the historical query information record table; the preset first screening condition includes: the cumulative number of queries of the query task within the first historical preset time period is greater than or equal to the preset number, and / or the single query time period is greater than or equal to the preset time period;

[0030] Adding the at least one second query task to be preprocessed to the preprocessing task information configuration table;

[0031] The second query task in the preprocessing task information configuration table is preprocessed.

[0032] Optionally, the preprocessing the second query task in the preprocessing task information configuration table includes:

[0033] For any second query task in the pre-processing task information configuration table, determine whether the parent task corresponding to the second query task is successfully executed according to the parent task status table of the first database; the parent task status table records the execution information of at least one parent task, and the parent task is used to update the data in the source data table to which the query result corresponding to the second query task belongs;

[0034] If it is determined that the parent task corresponding to the second query task is executed successfully, using the query statement corresponding to the second query task to obtain the second query result of the second query task from the data table corresponding to the parent task;

[0035] Using the second query result as the preprocessing result of the second query task, and generating a preprocessing result table corresponding to the second query task;

[0036] The preprocessing result table corresponding to the second query task is stored.

[0037] Optionally, storing the preprocessing result table corresponding to the second query task includes:

[0038] The preprocessing result table corresponding to the second query task is stored in the second database.

[0039] Optionally, after storing the preprocessing result table corresponding to the second query task, the method further includes:

[0040] The execution information of the second query task is added to the preprocessing task status table in the first database, and the execution information includes: the identifier of the preprocessing result table corresponding to the second query task, the storage location of the preprocessing result table corresponding to the second query task, the latest update time of the preprocessing result of the second query task, and whether to continue updating.

[0041] Optionally, the method further comprises:

[0042] According to the pre-processing task status table, a third query task that needs to continue pre-processing is screened out; the third query task satisfies a preset second screening condition, and a parent task corresponding to the third query task is updated regularly; the preset second screening condition includes: the cumulative number of queries of the historical query task within the second historical preset time length is greater than or equal to the preset number;

[0043] Obtaining the time of next preprocessing of the third query task;

[0044] When the time for the next preprocessing arrives, preprocessing the third query task, and updating the preprocessing result table corresponding to the third query task according to the preprocessing result of the third query task;

[0045] The execution information of the third query task is updated in the pre-processing task status table.

[0046] Optionally, the method further comprises:

[0047] The fourth query task which is no longer to be preprocessed is deleted from the preprocessing task status table; the fourth query task does not satisfy the preset second screening condition.

[0048] In a second aspect, the present application provides a data query device, the device comprising:

[0049] A first processing module, configured to generate a first query statement corresponding to a first query task from a client;

[0050] A second processing module is used to determine whether there is a first query result of the first query task in the cache according to the first query statement, and to obtain the first query result from the preprocessing result table corresponding to the first query task using the first query statement when the first query task exists in the preprocessing task information configuration table; the preprocessing task information configuration table records at least one preprocessed query task;

[0051] A sending module is used to send the first query result to the client.

[0052] Optionally, the second processing module is also used to, after determining whether the first query task exists in the pre-processing task information configuration table in the first database, use the first query statement to obtain the first query result from the source data table when the first query task does not exist in the pre-processing task information configuration table.

[0053] Optionally, the source data table is located in a second database.

[0054] Optionally, the second processing module is also used to, after the first processing module generates a first query statement corresponding to the first query task, use the first query statement to obtain the first query result from the cache when it is determined according to the first query statement that there is the first query result in the cache.

[0055] Optionally, the first processing module is specifically configured to concatenate and transform a plurality of query conditions included in the first query task to obtain the first query statement.

[0056] Optionally, the device further includes: a storage module, and the storage module is used to store the first query result in the cache according to the first query statement.

[0057] Optionally, the storage module is used to store first query information corresponding to the first query task in a historical query information record table of the first database, where the first query information includes: the first query statement and query time sub-information of the first query task.

[0058] Optionally, the second processing module is also used to determine whether the historical query information record table has the first query task before determining that there is no first query result of the first query task in the cache according to the first query statement, and when it is determined that the historical query information record table has the first query task, determine whether there is the first query result of the first query task in the cache according to the first query statement; or, when it is determined that there is no first query task in the historical query information record table, use the first query statement to obtain the first query result from the source data table.

[0059] Optionally, the second processing module is also used to filter at least one second query task to be preprocessed from the historical query information record table according to a preset first filtering condition; add the at least one second query task to be preprocessed to the preprocessing task information configuration table; and preprocess the second query task in the preprocessing task information configuration table; wherein the preset first filtering condition includes: the cumulative number of queries for historical query tasks within a first historical preset time period is greater than or equal to the preset number, and / or the duration of a single query is greater than or equal to the preset time period.

[0060] Optionally, the second processing module is specifically used to determine, for any second query task in the pre-processing task information configuration table, whether the parent task corresponding to the second query task is successfully executed according to the parent task status table of the first database; if it is determined that the parent task corresponding to the second query task is successfully executed, use the query statement corresponding to the second query task to obtain the second query result of the second query task from the data table corresponding to the parent task; use the second query result as the pre-processing result of the second query task, and generate and store the pre-processing result table corresponding to the second query task; wherein the parent task status table records the execution information of at least one parent task, and the parent task is used to update the data in the source data table to which the query result corresponding to the second query task belongs.

[0061] The storage module is further used to store the preprocessing result table corresponding to the second query task.

[0062] Optionally, the storage module is further used to store the preprocessing result table corresponding to the second query task in a second database.

[0063] Optionally, the second processing module is also used to add the execution information of the second query task to the preprocessing task status table in the first database after the storage module stores the preprocessing result table corresponding to the second query task, and the execution information includes: the identifier of the preprocessing result table corresponding to the second query task, the storage location of the preprocessing result table corresponding to the second query task, the latest update time of the preprocessing result of the second query task, and whether continued updating is required.

[0064] Optionally, the second processing module is also used to filter out the third query task that needs to continue preprocessing according to the preprocessing task status table; obtain the time of the next preprocessing of the third query task; when the time of the next preprocessing arrives, preprocess the third query task, and update the preprocessing result table corresponding to the third query task according to the preprocessing result of the third query task; update the execution information of the third query task in the preprocessing task status table; the third query task satisfies the preset second filtering condition, and the parent task corresponding to the third query task is updated periodically, wherein the preset second filtering condition includes: the cumulative number of queries of the query task within the second historical preset time period is greater than or equal to the preset number.

[0065] Optionally, the second processing module is further used to delete a fourth query task that is no longer performing preprocessing from the preprocessing task status table; and the fourth query task does not meet the preset second screening condition.

[0066] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0067] The memory stores computer-executable instructions;

[0068] The processor executes the computer-executable instructions stored in the memory to implement any of the above method steps.

[0069] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement any of the above method steps when executed by a processor.

[0070] The data query method, device, equipment and storage medium provided by the present application preprocess at least one query task and obtain the result corresponding to the query task. In this way, when a query task identical to the query task is subsequently received from the client, the result can be directly returned to the client without having to obtain the query result by executing the query task, thereby improving the query efficiency. When the query task is a time-consuming query task, the query efficiency of the data query system for querying time-consuming tasks can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0072] Figure 1 It is a structural schematic diagram of a data query system in the prior art;

[0073] Figure 2A schematic diagram of the structure of a data query system provided in an embodiment of the present application;

[0074] Figure 3 A flowchart of a data query method provided in an embodiment of the present application;

[0075] Figure 4 A flowchart of another data query method provided in an embodiment of the present application;

[0076] Figure 5 A flowchart of another data query method provided in an embodiment of the present application;

[0077] Figure 6 A schematic diagram of the structure of a data query device provided in an embodiment of the present application;

[0078] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0079] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0080] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0081] A data query system refers to a system that collects information from a database based on certain strategies and uses specific computer programs, organizes and processes the information, provides query services to users, and displays the relevant query information to users.

[0082] Figure 1 It is a structural diagram of a data query system in the prior art, such as Figure 1 As shown, the data query system in the prior art includes: a user layer, a service layer and a data storage layer.

[0083] The query task includes a query condition. Taking the data query system for storing the operator's user data as an example, the query condition may be: location, time, terminal device, area code, Internet access time, number of calls, historical information record table, call record, etc.

[0084] The service layer is usually implemented by a server cluster, which is used to convert the received query tasks into query statements, and after determining that there is no query result of the query task in the cache, it will use the query statement to query the data storage layer to obtain the query result, and feed the query result back to the client. The converted query statement matches the query statement supported by the database, for example, SQL statement.

[0085] The service layer is also provided with a cache for storing query results corresponding to recently executed query tasks. The specific storage quantity is related to the size of the cache and the size of the query results corresponding to the query tasks.

[0086] The data storage layer is provided with a MySQL database and a hive database, which are used to store the source data tables of the data query system. Each source data table includes at least one data. The data in these source data tables are the data stored by the data query system. Among them, the storage capacity of the MySQL database is relatively small, and it is used to store data that occupies a small storage space, and the storage capacity of the hive database is relatively large, and it is used to store data that occupies a large storage space.

[0087] At present, when the service layer performs data query, based on the received query task, if the query result is not found in the cache, it will also search in the data storage layer and send the query result to the client. However, with the advent of the big data era, the speed of data generation is growing explosively, and there are more and more time-consuming data query tasks. The use of the above data query method will cause the query time to become longer and longer, and even query timeouts will occur, which will lead to lower and lower query efficiency of the data query system.

[0088] In view of this, the present application provides a data query method that can pre-process at least one query task to obtain the query result of the at least one query task. In this way, when the query task is received from the client, the query result can be directly returned to the client without querying the database to obtain the query result, thereby improving the efficiency of data query. The above-mentioned pre-processed query task can be, for example, a time-consuming query task, a high-frequency query task, or a query task determined based on user needs, etc.

[0089] For ease of understanding, the following is an example of the application scenario of this application:

[0090] Figure 2 The structural diagram of the data query system provided for this application is as follows: Figure 2 As shown, the database may include: a user layer, a service layer and a data storage layer.

[0091] In this application, the user layer refers to a client located at the user side, which is used to receive a query task triggered by the user and send it to the service layer. The query task is a query task including a single query condition or a query task including multiple query conditions.

[0092] and Figure 1 The difference between the existing data query system shown is that the data storage layer stores a preprocessing task information configuration table, a preprocessing result table and a source data table. The preprocessing task information configuration table records at least one preprocessed query task. The preprocessing result table is used to store the query results of at least one preprocessed query task. The preprocessed query task mentioned here refers to a preprocessed query task, that is, a query task that has been pre-executed before receiving the query task sent by the client.

[0093] The data storage layer may include one or more databases. When the data storage layer includes multiple databases, the preprocessing task information configuration table and the preprocessing result table may be stored in the first database, or the preprocessing task information configuration table may be stored in the first database, and the preprocessing result table may be stored in the second database. The second database referred to herein is any database other than the first database.

[0094] It should be understood that the data storage layer includes multiple databases and can also have the function of storing source data tables, or in other words, the first database is only used to store the configuration table, and the other databases have the function of storing source data tables. When the second database stores both the preprocessing result table and the source data table, the storage area of ​​the preprocessing result table and the storage area of ​​the source data table are independent of each other, that is, located in different partitions of the second database.

[0095] Figure 2 This is a schematic diagram that takes the data storage layer as an example, including two databases (the first database and the second database, respectively). When the data storage layer includes two databases, the first database can be a MySQL database, and the second database can be, for example, a ClickHouse database. Among them, the MySQL database has a small storage capacity, stable performance, and stores data in the form of tables, which is convenient for storing and querying tables that occupy less storage space, such as preprocessing task information configuration tables. The ClickHouse database has a larger data storage capacity and a faster query speed. Storing the preprocessing result table and source data table that occupy a larger storage space in the ClickHouse database can improve data query efficiency.

[0096] After receiving the query task from the user layer, the service layer can determine whether there is a query result in the cache based on the query statement corresponding to the query task, and can check whether there is a first query task in the pre-processing task information configuration table in the first database based on the query statement. If there is, the service layer can directly obtain the query result from the pre-processing result table corresponding to the query task without querying the database again, thereby improving the query efficiency.

[0097] It should be understood that the functions of the service layer can be implemented by one server or by any server in a server cluster.

[0098] Optionally, the processing operation for the service layer can be implemented by the server or server cluster of the service layer. Alternatively, a server or server cluster is also set up in the data storage layer, and the server or server cluster of the service layer forwards the data to the server or server cluster of the data storage layer to perform data query operations.

[0099] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0100] Figure 3 The following is a flow chart of a data query method provided by this application. The execution subject of this method is a server at the service layer, such as Figure 3 As shown, the method comprises the following steps:

[0101] S301: Generate a first query statement corresponding to a first query task according to a first query task from a client.

[0102] In a possible implementation manner, the first query task may be composed of a single query condition, and the service layer converts the first query task into a corresponding first query statement.

[0103] In another possible implementation, the first query task may be composed of multiple query conditions, and the service layer parses and splices the first query task into a corresponding first query statement. Each query condition in the first query task is first converted into a query statement, and then spliced ​​in a preset order or format to obtain a first query statement. The first query statement obtained through splicing and conversion can query multiple options at the same time, so that the query condition can be selected independently and flexibly based on the user's query needs, thereby improving the flexibility of the query.

[0104] The query statements involved in this application are specifically determined by the query language supported by the database of the data query system, for example, they can be SQL statements.

[0105] S302: Determine whether there is a first query result of a first query task in the cache according to the first query statement.

[0106] In a possible implementation, when the service layer stores the historical query results corresponding to the historical query tasks in the cache in chronological order, the historical query results corresponding to each historical query task can be used as a cache file, and according to the historical query statement corresponding to the historical query task, the historical query statement can be mapped to the file name of the file through a hash algorithm to save the query results of different historical query tasks. Therefore, when judging whether there is a first query result of the first query task in the cache, the above-mentioned method of storing in the cache can also be used to convert the first query statement into a corresponding file name through a hash algorithm and then compare it with the file name of the historical query task in the cache to determine whether there is a first query result of the first query task in the cache.

[0107] If there is no file name identical to the file name corresponding to the first query statement in the cache, it means that there is no first query result of the first query task in the cache, indicating that the first query task may not have been queried before, so there is no first query result corresponding to the first query task in the cache, or, the first query task may have been queried before, but since the data in the cache will be updated, that is, the first query task is overwritten or deleted, there is no first query result corresponding to the first query task in the cache. At this time, it is necessary to search in the data storage layer, that is, execute step S303.

[0108] Optionally, if there is a file name in the cache that is the same as the file name corresponding to the first query statement, it means that there is a first query result of the first query task in the cache, which means that the query task has been queried before and recorded in the cache. At this time, the query result can be directly obtained from the cache to improve data query efficiency, that is, execute step S307.

[0109] S303: Determine whether there is a first query task in the pre-processing task information configuration table in the first database according to the first query statement.

[0110] It should be understood that the pre-processing task information configuration table stores query information corresponding to the pre-processing query task, wherein the query information may include: a query statement of the pre-processing query task and query time sub-information of the pre-processing query task.

[0111] It should be understood that the query time sub-information of the pre-processed query task may include: the start query time and the end query time of the query task, to record the time required to execute the query task. Alternatively, the query time sub-information of the pre-processed query task may include: the duration spent on executing the query task.

[0112] Exemplarily, taking the query time sub-information of the pre-processed query task including the start query time and the end query time of the query result query task as an example, the pre-processed task information configuration table may be shown in the following Table 1:

[0113] Table 1

[0114] Query Statement Start query time End query time Query statement 1 9:00 9:02 Query 2 10:10 10:15 Query statement 3 19:00 19:03 … … …

[0115] It should be understood that query statement 1, query statement 2 and query statement 3 represent three different query tasks, and the preprocessing task information configuration table can store query information of all preprocessing query tasks, and the preprocessing task information configuration table records at least one preprocessing query task.

[0116] Optionally, the query information corresponding to the preprocessed query task stored in the preprocessed task information configuration table includes, but is not limited to, the query statement of the preprocessed query task, the query time sub-information of the preprocessed query task, and may also include other information, such as the result quantity of the query result corresponding to the query task, the group by field, etc. Among them, the result quantity refers to the number of query results that can be queried by the query task, and the group by field refers to the query condition in the query statement.

[0117] Optionally, the service layer may compare the first query statement corresponding to the first query task with the query statements in the pre-processing task information configuration table one by one to determine whether the first query task exists.

[0118] If it is determined according to the first query statement that the first query task exists in the pre-processing task information configuration table in the first database, it indicates that the first query task is a pre-processing query task, and the query result of the first query task can be directly obtained, that is, step S304 is executed.

[0119] Optionally, if it is determined according to the first query statement that the first query task does not exist in the preprocessing task information configuration table in the first database, it indicates that the first query task has not been preprocessed and needs to be queried in the source data table storing all data information, that is, execute step S306.

[0120] S304: Use the first query statement to obtain a first query result from a preprocessing result table corresponding to the first query task.

[0121] Exemplarily, the preprocessing result table may be shown in the following Table 2, for example:

[0122] Table 2

[0123] Query Statement Preprocessing results Query statement 1 Preprocessing results 1 Query 2 Preprocessing result 2 Query statement 3 Preprocessing results 3 … …

[0124] It should be understood that query statement 1, query statement 2 and query statement 3 represent three different query tasks, and the preprocessing result table can store the query results of all query tasks that need to be preprocessed, and the preprocessing result table records at least one preprocessed query task.

[0125] Optionally, when obtaining the first query result from the preprocessing result table, the first query statement may be compared with the query statements in the preprocessing result table one by one to obtain the first query result corresponding to the first query statement.

[0126] S305: Send the first query result to the client.

[0127] Optionally, after obtaining the first query result, the first query result can be stored in a cache according to the first query statement, so that when the same query task occurs later, the query result can be directly obtained from the cache, thereby saving query time and improving query efficiency.

[0128] S306: Use the first query statement to obtain a first query result from the source data table.

[0129] After executing step S306, execute step S305.

[0130] It should be understood that the source data table is used to store all data stored in the data query system.

[0131] Regarding how to use the first query statement to obtain the first query result from the source data table, please refer to the implementation method of using the query statement to obtain the query result from the source data table in the prior art. For example, the query condition included in the query statement corresponding to the first query task can be compared with the original data in the source data table to filter out the data that meets the query condition as the query result.

[0132] S307: Use the first query statement to obtain a first query result from the cache.

[0133] Optionally, after determining that the cache has the first query result of the first query task according to the first query statement, the file may be directly read from the cache according to the file name to read the first query result from the file. After executing step S306, execute step S305.

[0134] Optionally, if the above-mentioned first query result is obtained from a preprocessing result table, or is obtained by querying from a source data table, after obtaining the first query result, the first query result can also be stored in the cache according to the first query statement, so that when the same query task exists in the future, the query result can be directly obtained in the cache, thereby saving query time and improving query efficiency.

[0135] Optionally, the data storage layer of the data query system may also store a historical query information record table, which is used to record the query information of the historical query tasks recently received from the client. The number of historical query tasks recorded in the historical query information record table is related to the size of the historical query information record table.

[0136] The query information of the historical query task may include: the historical query statement corresponding to the historical query task and the query time sub-information of the historical query task. When the data storage layer includes multiple databases, the historical query information record table may be stored in the first database or in the second database, which is not limited here.

[0137] Optionally, after obtaining the first query result of the first query task in the above method flow, the first query information corresponding to the first query task can also be stored in the historical query information record table of the first database, so as to facilitate the statistics of the query times of the historical query task. This application does not limit the specific execution time of storing the first query information corresponding to the first query task in the historical query information record table of the first database, as long as it is executed after obtaining the first query result.

[0138] In this implementation, optionally, before executing step S302, the following steps may also be included:

[0139] Determine whether there is a first query task in the historical query information record table. If it is determined that there is a first query task in the historical query information record table, it indicates that the first query result corresponding to the first query task has been searched before this query and may be stored in the cache or pre-processing result table, so step S302 can be continued. If it is determined that there is no first query task in the historical query information record table, it indicates that the first query result corresponding to the first query task may not have been searched before this query and cannot exist in the cache or pre-processing result table. Therefore, the step of querying the query result of the query task from the cache and pre-processing result table can be skipped, and step S306 can be directly executed to search from the source database to improve query efficiency.

[0140] Exemplarily, the first query statement may be compared with the query statements in the historical query information record table one by one to determine whether there is the first query task in the historical query information record table.

[0141] The present application pre-processes at least one query task and obtains the result corresponding to the query task. In this way, when a query task identical to the query task is subsequently received from the client, the result can be directly returned to the client without having to obtain the query result by executing the query task, thereby improving the query efficiency. When the query task is a time-consuming query task, the query efficiency of the data query system for querying time-consuming tasks can be improved.

[0142] The above content describes how to use the pre-processing task information configuration table. The following describes how to generate the pre-processing task information configuration table.

[0143] Figure 4 Another flow chart of a data query method provided by this application is as follows: Figure 4 As shown, the method comprises the following steps:

[0144] S401: Filter at least one second query task to be preprocessed from a historical query information record table according to a preset first filtering condition.

[0145] Optionally, the above-mentioned preset first screening condition can be specifically determined according to user needs.

[0146] Taking the pre-processed query task as a high-frequency task as an example, the preset first screening condition may include: the cumulative query times of the historical query task within the first historical preset time period is greater than or equal to the preset times. It should be understood that the cumulative query times within the first historical preset time period refers to the number of query information with the same query statement stored in the historical query information record table, wherein the first historical preset time period can be set according to demand and is not limited here.

[0147] Taking the pre-processed query task as a time-consuming task as an example, the preset first screening condition may include: the single query duration of the historical query task is greater than or equal to the preset duration.

[0148] Taking the pre-processed query task as a high-frequency and high-time-consuming task as an example, the preset first filtering condition can also include: the cumulative number of queries of the historical query task within the first historical preset time period is greater than or equal to the preset number, and the single query duration is greater than or equal to the preset duration.

[0149] Optionally, the preset first screening condition may further include: result quantity.

[0150] It should be understood that the second query tasks are the query tasks that are screened out and need to be preprocessed. These query tasks can be preprocessed only after the preset first screening condition is met, so as to be stored in the preprocessing result table.

[0151] S402: Add at least one second query task to be preprocessed to a preprocessing task information configuration table.

[0152] Optionally, the query tasks in the pre-processing task information configuration table may be obtained by screening out historical query tasks recorded in the historical query information recording table according to a preset first screening condition.

[0153] For example, adding the second query task to the preprocessing task information configuration table can be to add the query information of the second query task in the historical query information record table to the preprocessing task information configuration table to facilitate subsequent judgment on whether the preprocessing query task needs to continue the preprocessing operation.

[0154] It should be understood that, for the second query task, the query information of the second query task in the above-mentioned pre-processing task information configuration table can be the same as the query information of the second query task recorded in the historical query information recording table, or in other words, it is a subset or a complete set of the information in the historical query information recording table. Exemplarily, for the same query task, the query information of the query task included in the historical query information recording table includes a query statement and query time sub-information, then the pre-processing task information configuration table can include the query statement of the query task, or, include the query statement and query time sub-information.

[0155] S403: For any second query task in the pre-processing task information configuration table, determine whether the parent task of the second query task is executed successfully according to the parent task status table of the first database.

[0156] It should be understood that the parent task refers to a task in the data query system for regularly updating the source data table. The parent task status table records the execution status of the parent task in the data query system. The execution status can reflect whether the parent task is executed successfully, that is, whether the update operation on the source data table is completed.

[0157] Optionally, the parent task status table may include: parent task execution time and execution status. The parent task execution time also includes the start time and end time of the parent task execution, and the execution status may include execution start, execution in progress, and execution end. When the data storage layer includes multiple databases, the parent task status table may be stored in the first database, or in the second database, which is not limited here.

[0158] It should be understood that if the parent task corresponding to the second query task is not executed successfully, the execution information will be stored in the parent task status table. The service layer will send an alarm message to the terminal device held by the staff responsible for maintaining the data query system based on the execution status information in the parent task status table. The alarm message is used to remind the staff to inquire about the reason why the parent task was not executed successfully.

[0159] S404: Use the query statement corresponding to the second query task to obtain the second query result of the second query task from the data table corresponding to the parent task.

[0160] The data table corresponding to the parent task is the source data table updated by the parent task.

[0161] Regarding how to use the second query statement to obtain the second query result from the source data table updated by the parent task, please refer to the implementation method of using the query statement to obtain the query result from the source data table in the prior art. For example, the query condition included in the query statement corresponding to the second query task can be compared with the original data of the data in the source data table updated by the parent task to filter out the data that meets the query condition as the query result.

[0162] S405: Use the second query result as the preprocessing result of the second query task, and generate and store a preprocessing result table corresponding to the second query task.

[0163] It should be understood that generating a preprocessing result table corresponding to the second query task means writing the preprocessing result into the preprocessing result table, that is, writing the query statement and preprocessing result corresponding to the second query task into the preprocessing result table so as to obtain the query result in the preprocessing result table.

[0164] S406: Add the execution information of the second query task to the pre-processing task status table in the first database.

[0165] It should be understood that the preprocessing task status table records the execution information of the preprocessed query tasks that have completed the preprocessing operation. The execution information includes: the identifier of the preprocessing result table corresponding to the query task that has completed the preprocessing, the storage location of the preprocessing result table corresponding to the query task that has completed the preprocessing, the latest update time of the preprocessing result of the query task that has completed the preprocessing, and whether the preprocessing result table needs to be updated.

[0166] Adding the execution information of the second query task to the preprocessing task status table in the first database can include adding the execution information including the identifier of the preprocessing result table corresponding to the second query task, the storage location of the preprocessing result table corresponding to the second query task, the latest update time of the preprocessing result of the second query task, and whether the preprocessing result table needs to continue to be updated to the preprocessing task status table.

[0167] Optionally, there may be source data tables in the above data query system that need to be updated regularly. Therefore, in order to ensure the accuracy of the query results of the preprocessed query tasks, after the preprocessing of the query tasks in the preprocessing task status table is completed, the following operations may also be performed.

[0168] Figure 5 Another flow chart of a data query method provided by this application is as follows: Figure 5 As shown, the method comprises the following steps:

[0169] S501. Filter out the third query task that needs to continue preprocessing according to the preprocessing task status table.

[0170] It should be understood that the third query task is a query task that requires updating the query result according to the regularly updated data.

[0171] Exemplarily, the third query task selected above can meet the preset second screening condition, and the parent task corresponding to the third query task is updated regularly. The preset second screening condition includes: the cumulative query times of the query task within the second historical preset time period is greater than or equal to the preset times.

[0172] Optionally, the specific implementation method of screening the third query task can be as follows: according to the identifier of the preprocessing result table corresponding to the second query task in the preprocessing task status table, the total number of times the second query task is executed within the second historical preset time length in the historical query information record is counted. That is, when the cumulative number of queries within the second historical preset time length is greater than or equal to the preset number, the query task can be used as a candidate third query task.

[0173] It should be understood that the second historical preset time length here can be equal to the first historical preset time length, or can be greater than the first historical preset time length. The specific time length can be determined according to actual needs and is not limited here.

[0174] Then, based on the parent task status query table, determine whether the parent task corresponding to the candidate third query task needs to be executed regularly. If it needs to be executed regularly, it means that it is necessary to follow the parent task's update operation on the source data table and regularly update the query result of the candidate third query task, and then the candidate third query task is used as the third query task. If it does not need to be executed regularly, there is no need to update the query result of the candidate third query task.

[0175] S502: Obtain the time for the next preprocessing of the third query task.

[0176] It should be understood that the time for the third query task to perform the next preprocessing needs to be determined following the update operation of the parent task on the source data table.

[0177] For example, if the parent task corresponding to the third query task next executes an update operation on the source data table at 3:00, then the next preprocessing time of the third query task can be, for example, a moment postponed by a preset time interval based on 3:00, or it can be a moment obtained by postponing the execution of the parent task based on 3:00.

[0178] S503: When the time for the next preprocessing arrives, preprocess the third query task, and update the preprocessing result table corresponding to the third query task according to the preprocessing result of the third query task.

[0179] For how to preprocess the third query task, please refer to Figure 4The operation of preprocessing the second query task in will not be repeated here.

[0180] After obtaining the preprocessing result of the third query task, the preprocessing result can be used to replace the preprocessing result of the third query task recorded in the original preprocessing result table, or the two preprocessing results can be compared to update the difference part to the preprocessing result table corresponding to the third query task.

[0181] S504: Update the execution information of the third query task in the pre-processing task status table.

[0182] It should be understood that the specific implementation of step S504 can refer to Figure 4 The operation of preprocessing the second query task in will not be repeated here.

[0183] It should be noted that the above Figure 5 The operation shown may be a cyclically executed operation to avoid the problem of inaccurate query results of the preprocessed query task due to the query results of the source data table being updated periodically.

[0184] Optionally, if there is a fourth query task that does not meet the preset second screening condition in the preprocessing task status table, the fourth query task that is no longer preprocessed is deleted from the preprocessing task status table. The fourth query task may be any query task in the preprocessing task status table.

[0185] It should be understood that for the fourth query task that does not meet the preset second screening condition, it indicates that the fourth query task is no longer a query task that needs to be preprocessed. Taking the high-frequency query task as an example, when the cumulative query times of the third query task within the second historical preset time length are less than the preset times, it indicates that the query task has changed from a high-frequency query task to a low-frequency query task. At this time, no preprocessing operation can be performed on it to save storage space.

[0186] It should be understood that the server can perform the above-mentioned action of determining whether the historical query task needs to continue preprocessing when executing the query operation, or it can be decoupled from the query operation, that is, asynchronously processing the above-mentioned action of determining whether the historical query task needs to continue preprocessing, which has nothing to do with whether the server is currently executing the query operation from the client. When asynchronously executing, the server can be implemented through different threads, that is, one thread is used to execute the query operation from the client, and another thread is used to execute the above-mentioned Figure 4 , Figure 5 The query task preprocessing operations are shown in FIG.

[0187] The data query method provided in the embodiment of the present application can preprocess the query task after determining it as a preprocessed query task, so that when the data is queried again in the future, it can be directly queried in the preprocessing result table of the database, thereby improving the query rate.

[0188] In addition, the above-mentioned operations on each table and the operations on querying data from the source data table can be implemented by the server of the service layer directly accessing the database of the data storage layer, or the data storage layer can be deployed with a server cluster, and the server of the service layer interacts with the servers in the server cluster of the data storage layer, so that the servers in the server cluster of the data storage layer operate on the database to achieve the above-mentioned functions.

[0189] In the above embodiments, examples are given to illustrate the database where each table is located in the data query system. The following is an example to illustrate the database where each table is located in the data query system:

[0190] Assume that the data query system includes two databases, the first database is a MySQL database, which is used to store information such as the pre-processing task information configuration table, the historical query information record table, the pre-processing task status table, and the parent task status table. The second database is a ClickHouse database, which is used to store the pre-processing result table and the source data table.

[0191] The MySQL database has a small storage capacity, stable performance, and stores data in the form of tables, which is convenient for storing and querying tables that occupy a small storage space. Since the above configuration table stores the query information of the query task in the form of a table, when adding or deleting the query information in the configuration table, different configuration tables do not affect each other, so there is no need to modify the code or restart the application, and the purpose can be achieved by only modifying the corresponding configuration table.

[0192] The ClickHouse database has a larger data storage capacity and faster query speed. Its single-table query speed surpasses the popular databases and can quickly feedback the results to users. Therefore, the preprocessing result table and source data table that occupy a large storage space can be stored in the ClickHouse database, and the common queries, multi-table queries, time-consuming queries and other tasks stored in the database can be preprocessed to improve query efficiency.

[0193] When the client provides a query task, the service layer will stitch the query task into an SQL statement that matches the query statements supported by the database, and then search the historical query information record table for query information of the query task based on the SQL statement. By stitching the query task into an SQL statement, users can arbitrarily select query conditions on the client, such as time granularity, dimension, dimension selection condition, indicator and indicator screening condition, so as to flexibly query the required data.

[0194] If the query information of the query task exists in the historical query information record table, it means that the query task has been queried before, and it is necessary to search the cache for query results corresponding to the query task; if the query result exists in the cache, it is directly obtained and sent to the client; if the query result does not exist in the cache, it is necessary to find the query result corresponding to the query task in the preprocessing result table in the ClickHouse database to check whether the query task has been preprocessed in advance. If the query result exists in the preprocessing result table, it means that the query task has completed the preprocessing operation, and the query result can be directly obtained, saved to the cache, and sent to the client; if the query result does not exist in the preprocessing result table, it means that the query task may not have been preprocessed, and it is necessary to find the query result in the source data table in the ClickHouse database. After obtaining the query result, it is also necessary to save it to the cache and send it to the client. Among them, after obtaining the query result of the query task, the query information of the query task needs to be stored in the preprocessing task information configuration table.

[0195] This application will record different query conditions and query results in each step, so that when the user queries the same record again, the query results can be quickly responded to.

[0196] Based on the data query method provided in the above embodiment, the embodiment of the present application further provides an embodiment of a device for implementing each step and method in the above method embodiment.

[0197] Figure 6 A schematic diagram of a data query device provided in an embodiment of the present application is shown in FIG. Figure 6 The device includes: a first processing module 601, a second processing module 602, and a sending module 603. Optionally, the device may also include: a storage module 604.

[0198] A first processing module 601 is used to generate a first query statement corresponding to the first query task according to a first query task from a client;

[0199] The second processing module 602 is used to determine whether there is a first query result of the first query task in the cache according to the first query statement, and to obtain the first query result from the preprocessing result table corresponding to the first query task by using the first query statement when the first query task exists in the preprocessing task information configuration table. The preprocessing task information configuration table records at least one preprocessed query task.

[0200] The sending module 603 is used to send the first query result to the client.

[0201] Optionally, the second processing module 602 is also used to, after determining whether there is a first query task in the preprocessing task information configuration table in the first database, use the first query statement to obtain a first query result from the source data table when there is no first query task in the preprocessing task information configuration table.

[0202] Optionally, the source data table is located in the second database.

[0203] Optionally, the second processing module 602 is further used to, after the first processing module 601 generates a first query statement corresponding to the first query task, use the first query statement to obtain the first query result from the cache when it is determined that there is a first query result in the cache according to the first query statement.

[0204] Optionally, the first processing module 601 is specifically configured to concatenate and transform multiple query conditions included in the first query task to obtain a first query statement.

[0205] Optionally, the device further includes: a storage module 604, the storage module 604 is used to store the first query result in a cache according to the first query statement.

[0206] Optionally, the storage module 604 is used to store first query information corresponding to the first query task into a historical query information record table of the first database, where the first query information includes: a first query statement and query time sub-information of the first query task.

[0207] Optionally, the second processing module 602 is also used to determine whether the historical query information record table has the first query task before determining that there is no first query result of the first query task in the cache according to the first query statement, and when it is determined that the historical query information record table has the first query task, determine whether there is the first query result of the first query task in the cache according to the first query statement; or, when it is determined that there is no first query task in the historical query information record table, use the first query statement to obtain the first query result from the source data table.

[0208] Optionally, the second processing module 602 is also used to filter at least one second query task to be preprocessed from the historical query information record table according to a preset first filtering condition; add at least one second query task to be preprocessed to the preprocessing task information configuration table; and preprocess the second query task in the preprocessing task information configuration table; wherein the first filtering condition includes: the cumulative number of queries of the historical query task within the first historical preset time period is greater than or equal to the preset number, and / or the duration of a single query is greater than or equal to the preset time period.

[0209] Optionally, the second processing module 602 is specifically used to determine, for any second query task in the pre-processing task information configuration table, whether the parent task corresponding to the second query task is successfully executed according to the parent task status table of the first database; if it is determined that the parent task corresponding to the second query task is successfully executed, use the query statement corresponding to the second query task to obtain the second query result of the second query task from the data table corresponding to the parent task; use the second query result as the pre-processing result of the second query task, and generate and store the pre-processing result table corresponding to the second query task; wherein the parent task status table records the execution information of at least one parent task, and the parent task is used to update the data in the source data table to which the query result corresponding to the second query task belongs.

[0210] The storage module 604 is further used to store the preprocessing result table corresponding to the second query task.

[0211] Optionally, the storage module 604 is further configured to store the preprocessing result table corresponding to the second query task in the second database.

[0212] Optionally, the second processing module 602 is also used to add the execution information of the second query task to the preprocessing task status table in the first database after the storage module 604 stores the preprocessing result table corresponding to the second query task, and the execution information includes: the identifier of the preprocessing result table corresponding to the second query task, the storage location of the preprocessing result table corresponding to the second query task, the latest update time of the preprocessing result of the second query task, and whether continued updating is required.

[0213] Optionally, the second processing module 602 is also used to filter out the third query task that needs to continue preprocessing according to the preprocessing task status table; obtain the time of the next preprocessing of the third query task; when the time of the next preprocessing arrives, preprocess the third query task, and update the preprocessing result table corresponding to the third query task according to the preprocessing result of the third query task; update the execution information of the third query task in the preprocessing task status table; the third query task satisfies the preset second filtering condition, and the parent task of the third query task is updated periodically, wherein the preset second filtering condition includes: the cumulative number of queries of the query task within the second historical preset time period is greater than or equal to the preset number.

[0214] Optionally, the second processing module 602 is further used to delete the fourth query task that is no longer preprocessed from the preprocessing task status table; the fourth query task does not meet the preset second screening condition.

[0215] The device of the embodiment of the present application can be used to perform Figure 3-Figure 5 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be described in detail here. The device can be, for example, a server in a service layer or a chip in a server.

[0216] Figure 7 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 7 As shown, the electronic device may include: at least one processor 701 and a memory 702 .

[0217] The memory 702 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0218] The memory 702 may include a high-speed RAM memory, and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0219] The processor 701 is used to execute the computer-executable instructions stored in the memory 702 to implement the data query method described in the aforementioned method embodiment. The processor 701 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0220] Optionally, the electronic device may further include a communication interface. In a specific implementation, if the communication interface, the processor 701 and the memory 702 are implemented independently, the communication interface, the memory 702 and the processor 701 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0221] Optionally, in a specific implementation, if the communication interface, the processor 701 and the memory 702 are integrated on a chip, the communication interface, the processor 701 and the memory 702 can communicate through an internal interface.

[0222] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0223] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0224] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data query method, characterized in that: The method comprises: According to a first query task from a client, generating a first query statement corresponding to the first query task; If it is determined according to the first query statement that there is no first query result of the first query task in the cache, then determining according to the first query statement whether the first query task exists in a preprocessing task information configuration table in the first database, wherein the preprocessing task information configuration table records at least one preprocessing query task; If the first query task exists in the pre-processing task information configuration table, using the first query statement to obtain the first query result from the pre-processing result table corresponding to the first query task; Sending the first query result to the client; The method further comprises: storing first query information corresponding to the first query task in a historical query information record table of the first database, wherein the first query information includes: the first query statement and query time sub-information of the first query task; According to a preset first screening condition, at least one second query task to be preprocessed is screened from the historical query information record table; the preset first screening condition includes: the cumulative number of queries of the historical query task within the first historical preset time period is greater than or equal to the preset number, and / or the single query time period is greater than or equal to the preset time period; Adding the at least one second query task to be preprocessed to the preprocessing task information configuration table; For any second query task in the pre-processing task information configuration table, determine whether the parent task corresponding to the second query task is successfully executed according to the parent task status table of the first database; the parent task status table records the execution information of at least one parent task, and the parent task is used to update the data in the source data table to which the query result corresponding to the second query task belongs; If it is determined that the parent task corresponding to the second query task is executed successfully, using the query statement corresponding to the second query task to obtain the second query result of the second query task from the data table corresponding to the parent task; Using the second query result as the preprocessing result of the second query task, and generating a preprocessing result table corresponding to the second query task; The preprocessing result table corresponding to the second query task is stored.

2. The method according to claim 1, characterized in that: After determining whether the first query task exists in the pre-processing task information configuration table in the first database, the method further includes: If the first query task does not exist in the pre-processing task information configuration table, the first query statement is used to obtain the first query result from the source data table.

3. The method according to claim 2, characterized in that The source data table is located in the second database.

4. The method according to claim 1, characterized in that After generating the first query statement corresponding to the first query task, the method further includes: If it is determined according to the first query statement that the first query result exists in the cache, the first query statement is used to obtain the first query result from the cache.

5. The method according to any one of claims 1 to 4, characterized in that: The step of generating a first query statement corresponding to the first query task according to the first query task from the client includes: The multiple query conditions included in the first query task are concatenated and converted to obtain the first query statement.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: According to the first query statement, the first query result is stored in the cache.

7. The method according to claim 2, characterized in that: Before determining according to the first query statement that there is no first query result of the first query task in the cache, the method further includes: Determine whether the historical query information record table has the first query task; If it is determined that the historical query information record table has the first query task, determining whether the cache has the first query result of the first query task according to the first query statement; or, If it is determined that the historical query information record table does not have the first query task, using the first query statement to obtain the first query result from the source data table; The first query result is sent to the client.

8. The method according to claim 1, characterized in that The storing of the preprocessing result table corresponding to the second query task includes: The preprocessing result table corresponding to the second query task is stored in the second database.

9. The method according to claim 8, characterized in that After storing the preprocessing result table corresponding to the second query task, the method further includes: The execution information of the second query task is added to the preprocessing task status table in the first database, and the execution information includes: the identifier of the preprocessing result table corresponding to the second query task, the storage location of the preprocessing result table corresponding to the second query task, the latest update time of the preprocessing result of the second query task, and whether to continue updating.

10. The method according to claim 9, characterized in that The method further comprises: According to the pre-processing task status table, a third query task that needs to continue pre-processing is screened out; the third query task satisfies a preset second screening condition, and a parent task corresponding to the third query task is updated regularly; the preset second screening condition includes: the cumulative number of queries of the query task within the second historical preset time length is greater than or equal to the preset number; Obtaining the time of next preprocessing of the third query task; When the time for the next preprocessing arrives, preprocessing the third query task, and updating the preprocessing result table corresponding to the third query task according to the preprocessing result of the third query task; The execution information of the third query task is updated in the pre-processing task status table.

11. The method according to claim 10, characterized in that The method further comprises: The fourth query task which is no longer to perform preprocessing is deleted from the preprocessing task status table; the fourth query task does not meet the preset second screening condition.

12. A data query device, characterized in that: The device comprises: A first processing module, configured to generate a first query statement corresponding to a first query task from a client; A second processing module is used to determine whether the first query task exists in the preprocessing task information configuration table in the first database according to the first query statement after determining that there is no first query result of the first query task in the cache according to the first query statement, and when the first query task exists in the preprocessing task information configuration table, use the first query statement to obtain the first query result from the preprocessing result table corresponding to the first query task; if there is no first query task in the preprocessing task information configuration table, use the first query statement to obtain the first query result from the source data table; the preprocessing task information configuration table records at least one preprocessed query task; A sending module, used for sending the first query result to the client; A storage module, used to store first query information corresponding to the first query task into a historical query information record table of the first database, wherein the first query information includes: the first query statement and query time sub-information of the first query task; The second processing module is also used to filter at least one second query task to be preprocessed from the historical query information record table according to a preset first filtering condition; the preset first filtering condition includes: the cumulative number of queries of the historical query task within the first historical preset time period is greater than or equal to the preset number, and / or the single query time period is greater than or equal to the preset time period; the at least one second query task to be preprocessed is added to the preprocessing task information configuration table; for any second query task in the preprocessing task information configuration table, determine whether the parent task corresponding to the second query task is successfully executed according to the parent task status table of the first database; the parent task status table records the execution information of at least one parent task, and the parent task is used to update the data in the source data table to which the query result corresponding to the second query task belongs; if it is determined that the parent task corresponding to the second query task is successfully executed, the query statement corresponding to the second query task is used to obtain the second query result of the second query task from the data table corresponding to the parent task; the second query result is used as the preprocessing result of the second query task to generate a preprocessing result table corresponding to the second query task; and the preprocessing result table corresponding to the second query task is stored.

13. An electronic device, characterized in that: The electronic device comprises: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the data query method according to any one of claims 1 to 11 when executed by a processor.

15. A computer program product, characterized in that The computer program product stores computer-executable instructions, which, when executed by a processor, are used to implement the data query method according to any one of claims 1 to 11.

Images