Data query method and device, electronic equipment and storage medium

Abstract

The invention provides a data query method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a data query request, a label of the data query request and a to-be-executed query statement of the data query request; determining a target queue number according to a to-be-executed query statement of the data query request and preset queue cache information; according to the target queue number, the to-be-executed query statement is allocated to a target queue for query processing, and a query result is obtained; according to the query result, request processing state information is updated, target request processing state information is obtained, and the request processing state information comprises information of queried statements, query result information of the queried statements and information of unqueried statements; and determining target query result information from the target request processing state information according to the label of the data query request. According to the method, orderliness and accuracy of data query are guaranteed, and data query efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular, to a data query method, apparatus, electronic device, and storage medium. Background Art

[0002] The message session management system of an operator is mainly responsible for processing 4G and 5G online call records. During the business processing, a lot of business data is stored in the database. With the continuous development of information technology, the growth of business data shows an explosive trend, and a large amount of data that needs to be queried has been generated.

[0003] Currently, the existing data query method transmits and responds to data query requests through the connection links established by various business processing units and the database. Among them, the business processing unit sends a data query request to the database according to the business requirements, so that the database performs operations of adding, deleting, modifying, and querying data according to the data query request, and the business processing unit waits for the database to return the corresponding request response information.

[0004] However, the existing data query method has the problem of low data query efficiency. Summary of the Invention

[0005] This application provides a data query method, apparatus, electronic device, and storage medium to solve the problem of low data query efficiency existing in the existing data query method.

[0006] In a first aspect, this application provides a data query method, including:

[0007] Determine a data query request, a tag of the data query request, and a query statement to be executed for the data query request;

[0008] Determine a target queue number according to the query statement to be executed for the data query request and preset queue cache information, where the target queue number is the queue number of the target queue that can execute the query statement to be executed;

[0009] Allocate the query statement to be executed to the target queue for query processing according to the target queue number to obtain a query result, where the target queues corresponding to different target queue numbers are different, and different target queues are asynchronously connected to the target database corresponding to the data query request;

[0010] Update the request processing status information according to the query result to obtain target request processing status information, where the request processing status information includes information of the queried statements, query result information of the queried statements, and information of the unqueried statements;

[0011] Determine target query result information from the target request processing status information according to the tag of the data query request.

[0012] In an embodiment of the present application, determining a data query request, a tag of the data query request, and a query statement to be executed for the data query request includes:

[0013] Obtaining a client identifier of the client;

[0014] If the client identifier meets a preset authentication requirement, determining the client as a target client, where the authentication requirement is determined according to the client identifier and client information stored in the configuration information in the data query system;

[0015] Sending an authentication identifier to the target client, so that after the target client receives the authentication identifier, sending a data query request message to the data query system, where the data query request message includes the authentication identifier;

[0016] Determining the data query request, the tag of the data query request, and the query statement to be executed for the data query request according to the authentication identifier.

[0017] In an embodiment of the present application, before determining a target queue number according to the query statement to be executed for the data query request and preset queue cache information, the method further includes:

[0018] Determining a global queue in the data query system and the cache status of the global queue;

[0019] Determining an uncached queue in an uncached state and the queue number of the uncached queue according to the cache status of the global queue;

[0020] Configuring a service processing thread for the uncached queue, where the service processing thread is determined according to the data query request sent by the client;

[0021] Obtaining queue cache information according to the uncached queue, the queue number of the uncached queue, and the service processing thread of the uncached queue.

[0022] In an embodiment of the present application, determining a target queue number according to the query statement to be executed for the data query request and preset queue cache information includes:

[0023] Determining an identification number of the query statement to be executed and the quantity information of queues in the queue cache information;

[0024] Determining the target queue number according to the identification number of the query statement to be executed and the quantity information of queues in the queue cache information.

[0025] In an embodiment of the present application, updating request processing status information according to a query result to obtain target request processing status information includes:

[0026] Determining a query statement status table;

[0027] Determine the information of the queried statements, the query result information of the queried statements, and the information of the unqueried statements according to the query statement status table;

[0028] According to the query result, move the to-be-executed query statements in the information of the unqueried statements to the information of the queried statements, and update the query result information of the queried statements to obtain the target request processing status information.

[0029] In the embodiment of the present application, before determining the query statement status table, the method further includes:

[0030] Determine the initial query statement status table, the unqueried statement item, the queried statement item, and the queried statement result item in the initial query statement status table;

[0031] Update the unqueried statement item, the queried statement item, and the queried statement result item according to the historical data query request and the historical query result to obtain the query statement status table.

[0032] In the embodiment of the present application, updating the request processing status information according to the query result to obtain the target request processing status information includes:

[0033] Update the request processing status information according to the query result to obtain the initial target request processing status information;

[0034] Determine the number of unqueried statements, and the number of unqueried statements is determined according to the information of the unqueried statements in the initial target request processing status information;

[0035] According to the number of unqueried statements, return to execute the step of allocating the to-be-executed query statements to the target queue for query processing according to the target queue number to obtain the query result until the number of unqueried statements meets the preset quantity requirement.

[0036] In the embodiment of the present application, after determining the target query result information from the target request processing status information according to the label of the data query request, the method further includes:

[0037] Send the target query result information to the target client according to the label of the data query request.

[0038] In the embodiment of the present application, after sending the target query result information to the target client according to the label of the data query request, the method further includes:

[0039] Delete the content related to the data query request in the request processing status information according to the label of the data query request.

[0040] In a second aspect, the present application provides a data query device, including:

[0041] A first determination module, configured to determine a data query request, a tag of the data query request, and a query statement to be executed of the data query request;

[0042] A second determination module, configured to determine a target queue number according to the query statement to be executed of the data query request and preset queue cache information, where the target queue number is the queue number of a target queue that can execute the query statement to be executed;

[0043] A first obtaining module, configured to allocate the query statement to be executed to the target queue for query processing according to the target queue number, to obtain a query result, where the target queues corresponding to different target queue numbers are different, and different target queues are asynchronously connected to a target database corresponding to the data query request;

[0044] A second obtaining module, configured to update request processing status information according to the query result, to obtain target request processing status information, where the request processing status information includes information of query statements that have been executed, query result information of the query statements that have been executed, and information of query statements that have not been executed;

[0045] A third determination module, configured to determine target query result information from the target request processing status information according to the tag of the data query request.

[0046] In a third aspect, the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0047] The memory stores computer-executable instructions;

[0048] The processor executes the computer-executable instructions stored in the memory to implement the data query method of the embodiments of the present application.

[0049] In a fourth aspect, a computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the data query method of the embodiments of the present application.

[0050] The data query method, device, electronic device, and storage medium provided by this application determine a data query request, the tag of the data query request, and the query statement to be executed in the data query request; determine a target queue number according to the query statement to be executed in the data query request and the preset queue cache information, where the target queue number is the queue number of the target queue that can execute the query statement to be executed; allocate the query statement to be executed to the target queue for query processing according to the target queue number to obtain a query result, where the target queues corresponding to different target queue numbers are different, and different target queues are asynchronously connected to the target database corresponding to the data query request; update the request processing status information according to the query result to obtain the target request processing status information, where the request processing status information includes information on the query statements that have been queried, the query result information of the query statements that have been queried, and information on the query statements that have not been queried; determine the target query result information from the target request processing status information according to the tag of the data query request. When there are multiple query statements to be executed in the data query request, the queue numbers generated by using the query statements to be executed can be used to allocate each query statement to be executed to different queues, so that the threads in different queues process the query statements to be executed respectively, shortening the response time of data query, thereby improving the data query efficiency. Moreover, the queue numbers can ensure the orderliness of querying the query statements to be executed in each queue during data query, and at the same time, the tags of the data query requests can be used to combine the query results of each query statement to be executed in the data query request, ensuring the accuracy of the query results corresponding to the data query requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0052] Figure 1 It is a schematic flowchart of a data query method provided by an embodiment of this application;

[0053] Figure 2 It is a schematic flowchart of the service initialization process of a data query method provided by an embodiment of this application;

[0054] Figure 3 It is a schematic flowchart of the interface initialization process of a data query method provided by an embodiment of this application;

[0055] Figure 4 It is a schematic flowchart of the asynchronous operation processing process of a data query method provided by an embodiment of this application;

[0056] Figure 5 It is a schematic flowchart of the working thread processing process of a data query method provided by an embodiment of this application;

[0057] Figure 6 A structural schematic diagram of a data query device provided by an embodiment of the present application;

[0058] Figure 7 A structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0059] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and more detailed descriptions will be provided hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Specific Embodiments

[0060] Here, exemplary embodiments will be described in detail, and their examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0061] In the prior art, a connection link established by various service processing units and a database is used to transmit and respond to data query requests. Among them, the service processing unit sends a data query request to the database according to service requirements, so that the database performs operations of adding, deleting, modifying, and querying data according to the data query request, and the service processing unit waits for the database to return the corresponding request response information. However, when the connection link established by the service processing unit and the database sends a request for data query, it needs to wait for the database to return the query result before sending the next request. Therefore, before the database responds and returns the query result, the service processing unit will be in a blocked and inoperable state, resulting in the inability to process other service requests, thereby affecting the efficiency of data query.

[0062] To solve the above problems, the embodiments of the present application provide a data query method, device, electronic device, and storage medium, which can determine the query result of each request through the label of the data query request, and use threads in different queues to process the data query requests, realizing asynchronous query, so that the client can continue to send other query requests when it has not received the query result of the previous request, thereby solving the problem of client service blockage and improving the data query efficiency. At the same time, when there are multiple query statements to be executed in the data query request, the queue numbers generated by the query statements to be executed can be used to allocate each query statement to be executed to different queues, so that the threads in different queues process the query statements to be executed respectively, shortening the response time of data query, and ensuring the orderliness of querying the query statements to be executed in each queue during data query through the queue number.

[0063] The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] The embodiments of the present application provide a data query method, device, electronic device and storage medium, which are applied to a data query system. Here, the data query system is a data information interaction system between a client and a database. The data query system can define the connection interface between the client and the system through a protobuf (Google Protocol Buffers, protocol data exchange format) file, so that the client can implement the operation of data query by calling the connection interface. Among them, the client can be different types of business processing units, and the connection interface can be called by multiple clients. The type setting of the data query system in this embodiment is not particularly limited, as long as it can determine the data query request, the label of the data query request, and the query statement to be executed for the data query request; according to the query statement to be executed for the data query request and the preset queue cache information, determine the target queue number, where the target queue number is the queue number of the target queue that can execute the query statement to be executed; according to the target queue number, allocate the query statement to be executed to the target queue for query processing to obtain a query result, where the target queues corresponding to different target queue numbers are different, and different target queues are asynchronously connected to the target database corresponding to the data query request; according to the query result, update the request processing status information to obtain the target request processing status information, where the request processing status information includes information on the query statements that have been queried, the query result information of the query statements that have been queried, and information on the query statements that have not been queried; and determine the target query result information from the target request processing status information according to the label of the data query request.

[0065] Figure 1 The figure is a schematic flowchart of a data query method provided by an embodiment of the present application. The execution subject of this method can be a server storing the data query system or other servers. There is no special limitation here in this embodiment. As Figure 1 shown, this method may include:

[0066] S101. Determine the data query request, the label of the data query request, and the query statement to be executed for the data query request.

[0067] Among them, the data query request can be a request sent by the client for the database to perform operations such as data addition, deletion, modification, and query. The client can send multiple data query requests to the data query system; the label of the data query request can be used to distinguish each data query request; the query statement to be executed in the data query request can be an instruction statement set by the client for the database to perform corresponding operations. Among them, one data query request can include multiple query statements to be executed.

[0068] For example, in the embodiments of the present application, the data query request can refer to an SQL information array. The SQL information data array can be an array used to store SQL information, and the corresponding structure is: [SQL information 1, SQL information 2, SQL information 3]; the label of the data query request can refer to the batch number, that is, the batch corresponding to all query statements to be executed in the current data query request. This batch number is generated by the data query system and is used to return the query results according to the batch number. When the data query system obtains the SQL information, it will store the batch number in the corresponding SQL information; the query statement to be executed in the data query request can refer to the SQL information, where the SQL information can include an operation ID (used for client authentication and determining the target connection with the database), an SQL number (used to distinguish each query statement to be executed), an SQL identifier (used to determine the queue number corresponding to each query statement to be executed, ensuring that each query statement to be executed is executed in an orderly manner in different queues), an SQL statement (recording the instructions for data addition, deletion, modification, and query operations), the upper limit of the number of returned rows, the upper limit of the number of returned columns, and the batch number (before the data query system generates the corresponding batch number, this batch number information is empty).

[0069] Among them, in the embodiments of the present application, the method for determining the data query request, the label of the data query request, and the query statement to be executed in the data query request can include:

[0070] Obtain the client identifier of the client;

[0071] If the client identifier meets the preset authentication requirements, determine that the client is the target client. The authentication requirements are determined according to the client identifier and the client information stored in the configuration information of the data query system;

[0072] Send an authentication identifier to the target client, so that after the target client receives the authentication identifier, it sends a data query request message to the data query system, where the data query request message includes the authentication identifier;

[0073] According to the authentication identifier, determine the data query request, the label of the data query request, and the query statement to be executed in the data query request in the data query request message.

[0074] Among them, the client identifier may refer to the identity information sent by the client. The identity information may be the node ID corresponding to the client. The client information stored in the configuration information of the data query system may refer to the pre-stored client information array, which is used to establish a connection with the data query system. The client information array includes the client node IDs allowed to use the data query system and the port service IP for connecting to the data query system. The data query system can receive data query requests sent by different clients.

[0075] Among them, the client information stored in the configuration information of the data query system can also be used to generate service connection information. The service connection information may include the service connection handle established between the data query system and the client and the node ID of the client. Among them, the key in the service connection information is the node ID of the client, and the value is the service connection handle.

[0076] Among them, in the embodiment of the present application, the authentication identifier may refer to the operation ID. The generation method of the operation ID may be: when the data query system determines the client identifier sent by the client, increment the operation counter used to record the target client by 1 to obtain an operation count value, and generate the operation ID by using the current client identifier, the operation count value, and the time stamp of the current moment. The data query system will judge the data query request message sent by the client. Only the data query request message carrying the operation ID will be processed by the data query system.

[0077] Among them, in the embodiment of the present application, before sending the authentication identifier to the target client so that after the target client receives the authentication identifier, it sends a data query request message to the data query system, the method may further include:

[0078] Obtain the database connection string of the client and the maximum number of connections of the database. The database connection string is the usage information for connecting to the database, and the database connection string includes the database IP and port, the database name, the user name, and the connection password;

[0079] Generate operation information according to the client identifier, the database connection string, and the maximum number of connections of the database. The operation information is used to store the database connection pool handles corresponding to different clients.

[0080] Among them, the operation information can be used by the working thread to determine the corresponding database connection when executing the to-be-executed query statement, so as to return the data query result.

[0081] S102. Determine the target queue number according to the to-be-executed query statement of the data query request and the preset queue cache information, where the target queue number is the queue number of the target queue that can execute the to-be-executed query statement.

[0082] Among them, the queue cache information can be information configured according to the configuration information in the data query system, or information determined based on the initial queue cache information and the current queue cache status configured according to the configuration information in the data query system.

[0083] Among them, according to the configuration information in the data query system, the method for determining the initial queue cache information may include: constructing queues and working threads according to the number of concurrent working threads at system startup in the configuration information, allocating queue numbers to each queue to obtain the correspondence between the queue numbers and the queues; generating the initial queue cache information according to the correspondence between the queue numbers and the queues, and allocating corresponding working threads to each queue, so that the corresponding working threads perform data query operations according to the information of the statements to be executed cached in the queues. For example, if the number of concurrent working threads at system startup is n, then n queues and n working threads are constructed, and the queue numbers of each queue are allocated from 1 to n. Then, the key in the initial queue cache information is the queue number, and the value is the queue corresponding to the queue number.

[0084] Among them, in the embodiments of the present application, before determining the target queue number according to the statement to be executed in the data query request and the preset queue cache information, the method may further include:

[0085] Determining the global queue in the data query system and the cache status of the global queue;

[0086] Determining the uncached queues in the uncached state and the queue numbers of the uncached queues according to the cache status of the global queue;

[0087] Configuring the service processing threads of the uncached queues, and the service processing threads are determined according to the data query requests sent by the clients;

[0088] Obtaining the queue cache information according to the uncached queues, the queue numbers of the uncached queues, and the service processing threads of the uncached queues.

[0089] Among them, the global queue may be all the queues in the initial queue cache information, and the cache status may refer to whether there is information of statements to be executed stored in the queue. If there is no element corresponding to the information of the statements to be executed in the queue, it means that the queue is in the uncached state. If there is an element corresponding to the information of the statements to be executed in the queue, it means that the queue is in the cached state. Among them, whether there is an element corresponding to the information of the statements to be executed in the queue can be determined by listening through the working threads corresponding to the queues. Each working thread corresponds to listening to the elements in a queue, and the work of each working thread does not affect each other.

[0090] Among them, in the embodiments of the present application, the method for determining the target queue number according to the statement to be executed in the data query request and the preset queue cache information may include:

[0091] Determine the identification number of the query statement to be executed and the quantity information of the queues in the queue cache information;

[0092] Determine the target queue number according to the identification number of the query statement to be executed and the quantity information of the queues in the queue cache information.

[0093] Among them, the quantity information of the queues in the queue cache information may refer to the quantity of all uncached queues in the query system. The method for determining the target queue number may be to obtain the remainder by taking the modulus of the identification number of the query statement to be executed with the quantity information of the queues in the queue cache information. For example, if the identification number of the query statement to be executed is 3 and the quantity information of the queues in the queue cache information is 5, then 3 % 5 = 3 (% represents taking the modulus, that is, taking the remainder), and the queue number is 3.

[0094] S103. Allocate the query statement to be executed to the target queue for query processing to obtain a query result. Among them, the target queues with different target queue numbers are different, and different target queues are asynchronously connected to the target database corresponding to the data query request.

[0095] Among them, the method for allocating the query statement to be executed to the target queue for query processing may include: assembling the query statement to be executed and the label of the data query request into a queue element and caching it in the queue corresponding to the queue number; when the working thread corresponding to the queue monitors the queue element, take out the query statement to be executed corresponding to the queue element; determine the database connection pool handle in the operation information according to the operation ID in the query statement to be executed, so that the working thread can execute the query statement to be executed using the corresponding connection and obtain the query result from the database.

[0096] S104. Update the request processing status information according to the query result to obtain the target request processing status information, where the request processing status information includes the information of the queried statements, the query result information of the queried statements, and the information of the unqueried statements.

[0097] Among them, in the embodiments of the present application, the method for updating the request processing status information according to the query result to obtain the target request processing status information may include:

[0098] Determine the query statement status table;

[0099] Determine the information of the queried statements, the query result information of the queried statements, and the information of the unqueried statements according to the query statement status table;

[0100] Move the query statement to be executed in the information of the unqueried statements to the information of the queried statements according to the query result, and update the query result information of the queried statements to obtain the target request processing status information.

[0101] Among them, the queried statements can be all the executed and to-be-executed query statements corresponding to the tags of the data query requests, and the unqueried statements can be all the unexecuted and to-be-executed query statements corresponding to the tags of the data query requests.

[0102] Among them, in the embodiments of the present application, the query statement status table can be used to store request processing status information. The request processing status information can be the information for recording the query situation of the to-be-executed query statements in the data query system. The request processing status information can include the number of unqueried statements corresponding to the tags of the data query requests and the query result set. Among them, the query result set includes the queried statements corresponding to the tags of the data query requests and the corresponding relationship between the numbers of each queried statement and the query results. Among them, there can be multiple request processing status information in the query statement status table, and each request processing status information corresponds to a tag of a data query request, which is used to distinguish different data query requests.

[0103] Among them, in the embodiments of the present application, before determining the query statement status table, the method can further include:

[0104] Determine the initial query statement status table, including the unqueried statement item, the queried statement item, and the queried statement result item in the initial query statement status table;

[0105] Update the unqueried statement item, the queried statement item, and the queried statement result item according to the historical data query requests and historical query results to obtain the query statement status table.

[0106] Among them, the initial query statement status table can be constructed according to the data query request and the to-be-executed query statements of the data query request when determining the data query request message.

[0107] Among them, in the embodiments of the present application, the method for updating the request processing status information according to the query results to obtain the target request processing status information can include:

[0108] Update the request processing status information according to the query results to obtain the initial target request processing status information;

[0109] Determine the number of unqueried statements, and the number of unqueried statements is determined according to the information of the unqueried statements in the initial target request processing status information;

[0110] According to the number of unqueried statements, return to execute the step of allocating the to-be-executed query statements to the target queue for query processing according to the target queue number to obtain the query results until the number of unqueried statements meets the preset quantity requirements.

[0111] Among them, according to the query result, the method of updating the request processing status information can be to store the query result into the query result information of the queried statement in the query statement status table according to the number of the to-be-executed query statement corresponding to the query result, and subtract 1 from the number of unqueried statements in the information of unqueried statements to obtain the updated initial target request processing status information. The initial target request processing status information can be the information used to determine whether each to-be-executed query statement corresponding to the label of the data query request has been processed.

[0112] That the number of unqueried statements meets the preset quantity requirement can be when the number of unqueried statements is less than or equal to 0.

[0113] For example, the number of unqueried statements in the request processing status information is 2. After updating the information according to the query result A, the corresponding number of unqueried statements is updated to 1. At this time, it is necessary to return to execute the step of allocating the to-be-executed query statement to the target queue according to the target queue number for query processing to obtain the query result B. After updating the information according to the query result B, the corresponding number of unqueried statements is updated to 0. At this time, it can be determined that all the to-be-executed query statements corresponding to the label of the data query request have been processed, and then the step of allocating the to-be-executed query statement to the target queue according to the target queue number for query processing to obtain the query result is ended, and the target request processing status information is generated.

[0114] S105. Determine the target query result information from the target request processing status information according to the label of the data query request.

[0115] Among them, in the request processing status information, there is an association relationship between the label of the data query request and the corresponding query result information.

[0116] Among them, in the embodiment of the present application, after determining the target query result information from the target request processing status information according to the label of the data query request, the method may further include:

[0117] Send the target query result information to the target client according to the label of the data query request.

[0118] Among them, the method of sending the target query result information to the target client may include: determining the target client node ID and service connection handle from the service connection information configured by the data query system according to the configuration information according to the label of the data query request; calling the result return interface in the service connection information according to the corresponding service connection handle, and sending the target query result information to the target client.

[0119] Among them, in the embodiments of the present application, after sending the target query result information to the target client according to the tag of the data query request, the method may further include:

[0120] Delete the content related to the data query request in the request processing status information according to the tag of the data query request.

[0121] The data query method provided by the embodiments of the present application can determine the query result of each data query request through the tag of the data query request, which is convenient for the data query system to distinguish each data query request, so that the client can continue to send other query requests when it has not received the query result of the previous request; and use the working threads of different queues to process the data query requests to achieve asynchronous query, thereby solving the problem of client service blockage and improving the data query efficiency. At the same time, when there are multiple query statements to be executed in a data query request, the generated queue numbers of the query statements to be executed can be used to allocate each query statement to be executed to different queues, so that the threads in different queues process the query statements to be executed respectively, shortening the response time of the data query, and through the queue numbers, the orderliness of each queue processing the query statements to be executed during data query can be ensured.

[0122] Figure 2 It is a schematic diagram of the service initialization process of a data query method provided by the embodiments of the present application. As Figure 2 shown, the method may include:

[0123] (1) Initialize the operation counter and batch counter;

[0124] (2) Initialize the operation information map and SQL information cache queue map;

[0125] (3) Initialize the in-batch processing information map;

[0126] (4) Initialize the service information map of the business processing unit node;

[0127] (5) Create SQL information cache queues and working threads according to the total number of concurrent working threads in the configuration. For example, if the total number of concurrent working threads is configured as n, then n SQL information cache queues and n working threads need to be created. The queue numbers of each queue are allocated from 1 to n, and then the queue numbers are used as keys and the corresponding queues are used as values, and stored in the SQL information cache queue map, and each cache queue is assigned its own working thread;

[0128] (6) Traverse the array of business processing unit node information in the configuration. For each node, create a gRPC connection with the node based on the gRPC (Remote Procedure Calls) service IP and port in the node information to obtain a gRPC connection handle. Then, use the node ID in the node information as the key and the corresponding service operation handle as the value, and store them in the business processing unit node service information map.

[0129] (7) Start all worker threads according to the total number of concurrent worker threads in the configuration. Each worker thread is responsible for monitoring an SQL information cache queue.

[0130] Among them, in the embodiment of the present application, the operation counter is used to generate an operation ID; the batch counter is used to generate a batch number.

[0131] The operation information map can be used to store various operation information. The key is the operation ID, and the value is the operation information corresponding to the operation ID. For example, if there are three data query requests, the corresponding structure is: {operation ID1: operation information 1, operation ID2: operation information 2, operation ID3: operation information 3}; the operation information includes the client node ID and the corresponding database connection pool handle.

[0132] The SQL information cache queue map can be used to store various SQL information cache queues. The key is the queue number, and the value is the SQL information cache queue corresponding to the queue number. For example, if there are three query statements to be executed, the corresponding structure is: {"queue number 1": SQL information cache queue, "queue number 2": SQL information cache queue, "queue number 3": SQL information cache queue}; the SQL information cache queue includes the batch number and the corresponding SQL information; the SQL information can include the operation ID, SQL number (used to distinguish each SQL statement), SQL identifier (used to determine the queue number corresponding to each SQL statement to ensure that each SQL statement is executed in an orderly manner in different queues), SQL statement (a string recording the data addition, deletion, modification, and query operations), the upper limit of the number of returned rows, the upper limit of the number of returned columns, and the batch number.

[0133] The business processing unit node service information map can be used to store business processing unit node service information. The key is the node ID, and the value is the business processing unit gRPC connection handle corresponding to the node ID. For example, when there are three different clients (business processing units), the corresponding structure is: {"node ID1": grpc connection handle, "node ID2": grpc connection handle, "node ID3": grpc connection handle}.

[0134] Figure 3Schematic diagram of the interface initialization process of a data query method provided by an embodiment of the present application, as shown in Figure 3 shown, this method may include:

[0135] (1) Use an atomic operation to increment the operation counter by 1;

[0136] (2) Add a write lock to the operation information map;

[0137] (3) Generate an operation ID, and the generation method is: node ID + current timestamp + operation counter;

[0138] (4) Initialize the database connection pool handle according to the database connection string and the maximum number of database connections;

[0139] (5) Initialize the operation information, assign the node ID in the operation information to the node ID in the input parameter, and assign the database connection pool handle to the database connection pool handle initialized in step (4);

[0140] (6) Store the operation information in the operation information map, where the key is the operation ID generated in step (3), and the value is the operation information initialized in step (5);

[0141] (7) Release the write lock of the operation information map;

[0142] (8) Return the operation ID generated in step (3).

[0143] Figure 4 Schematic diagram of the asynchronous operation processing process of a data query method provided by an embodiment of the present application, as shown in Figure 4 shown, this method may include:

[0144] (1) Use an atomic operation to increment the batch counter by 1;

[0145] (2) Use the incremented batch counter as the batch number of this operation;

[0146] (3) Calculate the number of unexecuted SQL statements in this batch (i.e., the size of the SQL information array in the input parameter);

[0147] (4) Create a query result set map for this batch;

[0148] (5) Initialize the processing information within the batch using the number of unexecuted SQL statements in the batch and the query result set map;

[0149] (6) Add a write lock to the processing information map within the batch;

[0150] (7) Use the batch number as the key and store the processing information within the batch in the processing information map within the batch;

[0151] (8) Release the write lock of the in-batch processing information map;

[0152] (9) Traverse the incoming SQL information array. For each piece of SQL information, use the SQL identifier in the SQL information to take the modulus of the length of the SQL information cache queue map to obtain the queue number, and retrieve the queue corresponding to this queue number from the SQL information cache queue map. Assemble the batch number and the SQL information into a queue element and insert the element into the queue;

[0153] (10) Return.

[0154] Among them, the SQL information array can be an array used to store SQL information. For example, when there are three pieces of SQL information, the corresponding structure is: [SQL information 1, SQL information 2, SQL information 3].

[0155] The query result set map can be used to store the query results of SQL statements. The key is the SQL number, and the value is the array of query result rows corresponding to this SQL number. The corresponding structure is: {SQL number 1: array of query result rows, SQL number 2: array of query result rows, SQL number 3: array of query result rows}; The array of query result rows can be an array used to store multiple row records in a certain query result. The array elements are multiple column values in the row record (i.e., the query result column array). The corresponding structure is: [query result column array 1, query result column array 2, query result column array 3]; The query result column array can be an array used to store multiple column values in a single row record in a certain query result. The corresponding structure is: [column value 1, column value 2, column value 3].

[0156] The in-batch processing information map can be used to store the number of unexecuted SQL statements within a batch number. The key is the batch number, and the value is the number of unexecuted SQLs within this batch number. The structure is as follows: {batch number 1: in-batch processing information, batch number 2: in-batch processing information, batch number 3: in-batch processing information}; The in-batch processing information can include the number of unexecuted SQL statements and the query result set map. Among them, the number of unexecuted SQL statements can be the number of unexecuted SQL statements in the current batch.

[0157] Figure 5 It is a schematic diagram of the working thread processing flow of a data query method provided by an embodiment of the present application. As Figure 5 shown, the method may include:

[0158] (1) Check whether there are elements in the SQL information cache queue responsible for this thread. If not, jump to step (2). If so, retrieve an element from the queue (the element contains the batch number and SQL information), and jump to step (3);

[0159] Suspend for 1 second and jump to step (1);

[0160] (3) Add a read lock to the operation information map;

[0161] (4) Use the operation ID in the SQL information of the element to retrieve the corresponding operation information from the operation information map, and then retrieve the node ID and database connection pool handle from the operation information;

[0162] (5) Unlock the read lock of the operation information map;

[0163] (6) Use the database connection pool handle to obtain a connection, and use this connection to execute the SQL statement in the SQL information. According to the returned query records, as well as the upper limit of the number of returned columns and the upper limit of the number of returned rows in the SQL information, fill the query results into the query result row array;

[0164] (7) Add a write lock to the in-batch processing information map;

[0165] (8) Use the batch number in the element to retrieve the corresponding in-batch processing information from the in-batch processing information map. Insert the query result row array in step (6) into the query result set map in the in-batch processing information, where the key is the SQL number in the SQL information. Then subtract 1 from the number of unexecuted SQLs in the in-batch processing information. If the number of unexecuted SQLs after subtraction > 0, it means there are still unexecuted SQLs in this batch, then jump to step (10). If the number of unexecuted SQLs after subtraction <= 0, then jump to step (9);

[0166] (9) Use the node ID to retrieve the corresponding gRPC connection handle from the business processing unit node service information map. Use this handle to call the result set return interface (resSetReturn) provided by the business processing unit, send the query result set map to the business processing unit, and then clear the processing information of this batch number from the in-batch processing information map;

[0167] (10) Unlock the write lock of the in-batch processing information map;

[0168] (11) Jump to step (1).

[0169] The data query method provided by the embodiments of the present application can support asynchronous operations of single and multiple SQLs, and can ensure the orderliness of the execution of multiple SQLs. At the same time, as a service system, the data query system supports the access of multiple business processing units and ensures that the query operations of each business processing unit do not affect each other.

[0170] Figure 6The structural schematic diagram of the data query device provided by the embodiment of the present application is as follows. As Figure 6 shown, the data query device 60 includes: a first determination module 601, a second determination module 602, a first obtaining module 603, a second obtaining module 604, and a third determination module 605. Among them:

[0171] The first determination module 601 is configured to determine a data query request, a tag of the data query request, and a query statement to be executed for the data query request;

[0172] The second determination module 602 is configured to determine a target queue number according to the query statement to be executed for the data query request and the preset queue cache information, where the target queue number is the queue number of the target queue that can execute the query statement to be executed;

[0173] The first obtaining module 603 is configured to allocate the query statement to be executed to the target queue for query processing according to the target queue number to obtain a query result, where the target queues corresponding to different target queue numbers are different, and different target queues are asynchronously connected to the target database corresponding to the data query request;

[0174] The second obtaining module 604 is configured to update the request processing status information according to the query result to obtain the target request processing status information, where the request processing status information includes information on the query statements that have been executed, query result information on the query statements that have been executed, and information on the query statements that have not been executed;

[0175] The third determination module 605 is configured to determine the target query result information from the target request processing status information according to the tag of the data query request.

[0176] In the embodiment of the present application, the first determination module 601 may further be configured to:

[0177] Obtain the client identifier of the client;

[0178] If the client identifier meets the preset authentication requirements, determine that the client is the target client, and the authentication requirements are determined according to the client identifier and the client information stored in the configuration information of the data query system;

[0179] Send an authentication identifier to the target client, so that after the target client receives the authentication identifier, it sends a data query request message to the data query system, where the data query request message includes the authentication identifier;

[0180] Determine the data query request, the tag of the data query request, and the query statement to be executed for the data query request in the data query request message according to the authentication identifier.

[0181] In the embodiment of the present application, the second determination module 602 may further be configured to:

[0182] Determine the global queue in the data query system and the cache status of the global queue;

[0183] According to the cache status of the global queue, determine the uncached queue in the uncached state and the queue number of the uncached queue;

[0184] Configure the service processing thread of the uncached queue, and the service processing thread is determined according to the data query request sent by the client;

[0185] Obtain queue cache information according to the uncached queue, the queue number of the uncached queue, and the service processing thread of the uncached queue.

[0186] In the embodiment of the present application, the second determination module 602 can also be used for:

[0187] Determine the identification number of the query statement to be executed and the quantity information of the queues in the queue cache information;

[0188] According to the identification number of the query statement to be executed and the quantity information of the queues in the queue cache information, determine the target queue number.

[0189] In the embodiment of the present application, the second obtaining module 604 can also be used for:

[0190] Determine the query statement status table;

[0191] According to the query statement status table, determine the information of the queried statements, the query result information of the queried statements, and the information of the unqueried statements;

[0192] According to the query result, move the query statement to be executed in the information of the unqueried statements to the information of the queried statements, and update the query result information of the queried statements to obtain the target request processing status information.

[0193] In the embodiment of the present application, the second obtaining module 604 can also be used for:

[0194] Determine the initial query statement status table, the unqueried statement item, the queried statement item, and the queried statement result item in the initial query statement status table;

[0195] Update the unqueried statement item, the queried statement item, and the queried statement result item according to the historical data query request and the historical query result to obtain the query statement status table.

[0196] In the embodiment of the present application, the second obtaining module 604 can also be used for:

[0197] Update the request processing status information according to the query result to obtain the initial target request processing status information;

[0198] Determine the number of unqueried statements, where the number of unqueried statements is determined according to the information of the unqueried statements in the initial target request processing status information;

[0199] According to the number of unqueried statements, return and execute the step of allocating the to-be-executed query statements to the target queue for query processing according to the target queue number to obtain the query result until the number of unqueried statements meets the preset quantity requirement.

[0200] In the embodiment of the present application, the third determination module 605 can also be used for:

[0201] Send the target query result information to the target client according to the label of the data query request.

[0202] In the embodiment of the present application, the third determination module 605 can also be used for:

[0203] Delete the content related to the data query request in the request processing status information according to the label of the data query request.

[0204] Figure 7 It is a schematic structural diagram of the electronic device provided in the embodiment of the present application. As Figure 7 shown, the electronic device 70 includes:

[0205] The electronic device 70 may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, and a network interface 703. Among them, the processor 701, the memory 702, and the network interface 703 are connected through a bus 704.

[0206] In the specific implementation process, at least one processor 701 executes the computer execution instructions stored in the memory 702, so that at least one processor 701 executes the above data query method.

[0207] For the specific implementation process of the processor 701, reference can be made to the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0208] In the above Figure 7In the illustrated embodiments, it should be understood that the processor may be a central processing unit (CPU for short), or other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly implemented by the execution of a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0209] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0210] The network interface may be a wireless network interface or a wired network interface, which is generally used to establish a communication connection between electronic devices. For example, the network interface is used to connect the electronic device to an external terminal through a network, and establish a data transmission channel and a communication connection between the electronic device and the external terminal, etc.

[0211] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0212] In some embodiments, a computer program product is also proposed, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps in any of the above data query methods are implemented.

[0213] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware through instructions. The instructions can be stored in any computer-readable storage medium and loaded and executed by a processor.

[0214] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions that can be loaded by a processor to execute the steps in any of the data query methods provided by the embodiments of the present application.

[0215] Among them, the storage medium may include: Read Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disc, etc.

[0216] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.

[0217] Since the instructions stored in the storage medium can execute the steps in any of the data query methods provided by the embodiments of the present application, the beneficial effects that can be achieved by any of the data query methods provided by the embodiments of the present application can be realized. For details, see the previous embodiments and will not be repeated here.

[0218] For the specific implementation of each of the above operations, reference may be made to the previous embodiments and will not be repeated here.

[0219] Those skilled in the art will readily conceive of other implementations of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0220] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A data query method, characterized in that, Applied to a data query system, including: Determine a data query request, a tag of the data query request, and a query statement to be executed for the data query request; Determine a target queue number according to the query statement to be executed for the data query request and preset queue cache information, where the target queue number is the queue number of a target queue that can execute the query statement to be executed; Allocate the query statement to be executed to the target queue for query processing according to the target queue number to obtain a query result, where the target queues corresponding to different target queue numbers are different, and different target queues are asynchronously connected to a target database corresponding to the data query request; Update the request processing status information according to the query result to obtain target request processing status information, where the request processing status information includes information on query statements that have been executed, query result information of the executed query statements, and information on query statements that have not been executed; Determine target query result information from the target request processing status information according to the tag of the data query request.

2. The method according to claim 1, characterized in that, The determining the data query request, the tag of the data query request, and the query statement to be executed for the data query request includes: Obtain a client identifier of the client; If the client identifier meets a preset authentication requirement, determine that the client is a target client, where the authentication requirement is determined according to the client identifier and client information stored in the configuration information of the data query system; Send an authentication identifier to the target client, so that after the target client receives the authentication identifier, it sends a data query request message to the data query system, where the data query request message includes the authentication identifier; Determine the data query request, the tag of the data query request, and the query statement to be executed for the data query request in the data query request message according to the authentication identifier.

3. The method according to claim 1, characterized in that, Before determining the target queue number according to the query statement to be executed for the data query request and the preset queue cache information, the method further includes: Determine a global queue in the data query system and a cache status of the global queue; Determine an uncached queue in an uncached state and a queue number of the uncached queue according to the cache status of the global queue; Configure a service processing thread for the uncached queue, where the service processing thread is determined according to a data query request sent by the client; Obtain the queue cache information according to the uncached queue, the queue number of the uncached queue, and the service processing thread of the uncached queue.

4. The method according to claim 1, characterized in that, The determining the target queue number according to the query statement to be executed for the data query request and the preset queue cache information includes: Determine an identification number of the query statement to be executed and quantity information of queues in the queue cache information; Determine the target queue number according to the identification number of the query statement to be executed and the quantity information of queues in the queue cache information.

5. The method according to claim 1, characterized in that, The updating the request processing status information according to the query result to obtain the target request processing status information includes: Determine a query statement status table; Determine the information of the queried statement, the query result information of the queried statement, and the information of the unqueried statement according to the query statement status table; According to the query result, move the to-be-executed query statement in the information of the unqueried statement to the information of the queried statement, and update the query result information of the queried statement to obtain the target request processing status information.

6. The method according to claim 5, characterized in that, Before determining the query statement status table, the method further includes: Determine an initial query statement status table, including the unqueried statement item, the queried statement item, and the queried statement result item in the initial query statement status table; Update the unqueried statement item, the queried statement item, and the queried statement result item according to the historical data query request and the historical query result to obtain the query statement status table.

7. The method according to claim 1, characterized in that, The updating the request processing status information according to the query result to obtain the target request processing status information includes: Update the request processing status information according to the query result to obtain the initial target request processing status information; Determine the number of unqueried statements, where the number of unqueried statements is determined according to the information of the unqueried statements in the initial target request processing status information; According to the number of unqueried statements, return to execute the step of allocating the to-be-executed query statement to the target queue for query processing to obtain the query result until the number of unqueried statements meets the preset quantity requirement.

8. The method according to claim 1, characterized in that, After determining the target query result information from the target request processing status information according to the label of the data query request, the method further includes: Send the target query result information to the target client according to the label of the data query request.

9. The method according to claim 8, wherein After sending the target query result information to the target client according to the label of the data query request, the method further includes: Delete the content related to the data query request in the request processing status information according to the label of the data query request.

10. A data query device, characterized in that Includes: A first determination module, configured to determine a data query request, the label of the data query request, and the to-be-executed query statement of the data query request; A second determination module, configured to determine a target queue number according to the to-be-executed query statement of the data query request and the preset queue cache information, where the target queue number is the queue number of the target queue that can execute the to-be-executed query statement; A first obtaining module, configured to allocate the to-be-executed query statement to the target queue for query processing according to the target queue number to obtain a query result, where the target queues corresponding to different target queue numbers are different, and different target queues are asynchronously connected to the target database corresponding to the data query request; A second obtaining module, configured to update the request processing status information according to the query result to obtain the target request processing status information, where the request processing status information includes the information of the queried statement, the query result information of the queried statement, and the information of the unqueried statement; A third determination module, configured to determine target query result information from the target request processing status information according to the label of the data query request.

11. An electronic device, characterized in that Comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the data query method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the data query method according to any one of claims 1 to 9.

Images